Description

You will design and maintain real-time data pipelines on Google Cloud Platform. You own the end-to-end implementation of streaming jobs using Apache Beam, Dataflow, and Pub/Sub.

This role is hybrid.

Responsibilities

• Develop and deploy real-time streaming data pipelines using Apache Beam with Java and Python.
• Manage data ingestion and processing workflows via Google Cloud Dataflow and Pub/Sub.
• Orchestrate scheduled and event-driven jobs using Cloud Composer.
• Store and query large-scale datasets in BigQuery.
• Integrate Apache Kafka for robust message streaming.

Required Skills

• 10+ years of experience in data engineering or related fields.
• Strong proficiency in Java and Python for data processing.
• Hands-on experience with Google Cloud Platform services: Dataflow, Pub/Sub, BigQuery, and Composer.
• Deep understanding of Apache Beam programming model.
• Experience implementing real-time streaming architectures.
• Working knowledge of Apache Kafka.
• Any Graduate degree.

Preferred Skills

• Experience optimizing BigQuery queries for performance and cost.
• Familiarity with CI/CD pipelines for data infrastructure.

Key Skills

Education

Any Graduate