Description

Design and optimize BigQuery data models, schemas, and storage for performance and cost efficiency. Build scalable ELT/ETL pipelines using Cloud Composer, Dataflow, and Dataproc.

This role is on-site.

Responsibilities

• Design and optimize BigQuery data models, schemas, and storage for performance and cost efficiency.
• Build scalable ELT/ETL pipelines and workflows using Cloud Composer (Airflow), Dataflow, and Dataproc.
• Write complex SQL scripts and tune query performance to minimize BigQuery slot usage.
• Ingest, transform, and manage data from diverse sources into BigQuery, including orchestration with Pub/Sub.
• Implement governance and security controls, including access management and best practices for data protection.

Required Skills

• 6-12 years of experience in cloud data engineering, data warehousing, and ETL/ELT pipeline development.
• Hands-on experience with Google Cloud Platform (GCP) services, specifically BigQuery and GoogleSQL.
• Proficiency in Python for data processing and pipeline development.
• Experience with data processing frameworks such as Apache Spark, Apache Beam, or Hadoop.
• SQL expertise with orchestration/deployment experience using Apache Airflow and/or Cloud Composer.
• Experience with streaming/batch integration using Pub/Sub, Dataflow, and Cloud Storage.

Preferred Skills

• BigQuery-specific optimization and advanced modeling patterns for large datasets.

Key Skills

Education

Any Graduate