Description

You build and maintain scalable batch and real-time data pipelines.

Responsibilities

• Design, build, and maintain scalable batch and real-time data pipelines using Apache Spark (PySpark), SQL, and Hive.
• Develop ELT/ETL workflows to ingest, transform, and load enterprise data into BigQuery data warehouses.
• Design and implement cloud-native data architectures on GCP, integrating transactional and analytical platforms.
• Optimize Spark jobs, SQL queries, and BigQuery workloads to improve performance and reduce processing latency.
• Develop and manage workflow orchestration using Apache Airflow (Cloud Composer) for ingestion, validation, and loading.

Required Skills

• 5+ years of experience in data engineering or big data development.
• Strong expertise with Google Cloud Platform (GCP).
• Deep experience with Apache Spark and BigQuery.
• Proficiency in SQL and Hive.
• Experience building data ingestion frameworks using GCP services (Cloud Dataflow, Pub/Sub, GCS).
• Experience designing dimensional data models (fact and dimension tables).
• Knowledge of data governance, lineage, and quality checks in pipelines.
• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or related field.

Key Skills

Education

Bachelor's or Master's degrees