Description

You will lead the design and implementation of large-scale data engineering solutions.

Responsibilities

• Design and optimize data ingestion and modeling processes for petabyte-scale environments.
• Orchestrate complex data pipelines using workflow management tools.
• Implement and maintain CI/CD pipelines and automation for data workflows.
• Ensure data platform reliability by implementing monitoring and operational automation.
• Collaborate with stakeholders to translate business needs into technical data solutions.

Required Skills

• 10+ years of overall IT experience.
• Expert-level proficiency in Python, including data engineering libraries.
• Advanced SQL for complex querying and performance tuning.
• Extensive hands-on experience with Google Cloud Platform (GCP) services.
• Proven experience with data pipeline orchestration tools (e.g., Apache Airflow, Cloud Composer).
• Proficiency with CI/CD pipelines and version control (Git).
• Familiarity with tools like GitLab CI/CD, GitHub Actions, or Jenkins.
• Experience with data processing frameworks like Cloud Dataflow/Beam or Pub/Sub.
• Knowledge of Docker and Kubernetes is required or preferred.

Preferred Skills

Key Skills

Education

Any Graduate