Description

You will design and maintain data pipelines on Google Cloud Platform using BigQuery, Dataflow, and Composer.

Responsibilities

• Build and optimize ETL workflows using Python, SQL, and Spark.
• Implement CI/CD pipelines with Jenkins and GitHub for automated testing and deployment.
• Manage data modeling, mapping, and NoSQL database integrations.
• Coordinate onsite-offshore teams and apply Agile and DevOps methodologies.
• Develop API scripts and leverage Google Cloud SDK for infrastructure automation.

Required Skills

• 5+ years of experience in data engineering roles.
• Strong proficiency in Python and SQL.
• Hands-on experience with Google Cloud services: BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
• Proficiency in Apache Hadoop ecosystem (Hive, MapReduce, Spark).
• Experience with Airflow for DAG management and orchestration.
• Expertise in Git version control and CI/CD tools like Jenkins.
• Familiarity with NoSQL databases and data modeling concepts.
• Experience with Test Driven Development (TDD).
• Programming experience in Java or Scala.

Preferred Skills

• Google Cloud Professional Data Engineer Certification.
• Domain experience in Retail.

Key Skills

Education

Not specified