Description

You will design, develop, and maintain batch and streaming data pipelines primarily on Google Cloud Platform.

Responsibilities

• Build, optimize, and manage BigQuery datasets, tables, views, and complex SQL queries.
• Develop data pipelines using Dataflow (Apache Beam), Pub/Sub, and Cloud Storage.
• Perform data ingestion from diverse sources including APIs, databases, and files.
• Ensure data accuracy, consistency, validation, monitoring, and robust error handling.
• Automate and schedule workflows using Cloud Composer (Airflow).

Required Skills

• Strong experience with Google Cloud Platform and its data engineering services.
• Hands-on experience with the GCP data stack: BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
• Strong understanding of batch and streaming data processing architectures.
• Experience optimizing SQL queries within BigQuery.
• Hands-on experience ingesting data from APIs, databases, and flat files.
• Experience implementing IAM, security controls, and compliance standards in cloud environments.
• Proficiency with workflow orchestration tools like Cloud Composer (Airflow).
• Understanding of cloud performance optimization and cost management.
• Ability to collaborate effectively with cross-functional teams.

Key Skills

Education

Any Graduate