Data Engineer

Design, build, and maintain scalable data pipelines on Google Cloud Platform using Python and SQL.

This role is on-site.

Responsibilities

• Architect and operate reliable batch and near real-time workflows using Apache Airflow and Cloud Composer.
• Develop ELT/ETL processes leveraging BigQuery, Dataflow, Pub/Sub, and Cloud Storage for efficient data ingestion.
• Write expert-level SQL for complex transformations, windowing functions, and performance tuning to optimize query efficiency and costs.
• Build Python data components, including operators and hooks, and manage CI/CD pipelines for deployment.
• Implement data quality checks, SLAs, and documentation standards while partnering with product and analytics teams.

Required Skills

• 5+ years of experience with SQL, including performance tuning and complex modeling.
• 5+ years of Python programming for data engineering tasks.
• 3+ years of experience with Apache Airflow and Cloud Composer.
• 3+ years of hands-on experience with GCP services (BigQuery, Dataflow, Pub/Sub, GCS, Composer).
• Strong understanding of Big Data concepts, MPP warehouses, and warehousing/processing performance.
• Experience with Postgres for data sources and targets.
• Proficiency in CI/CD practices for data deployments.
• Bachelor's degree in Computer Science or related field.

Preferred Skills

• 10+ years of total professional experience in data engineering or related roles.
• Experience with schema evolution and partitioning/clustering strategies.