Description

You will build and maintain end-to-end data pipelines within a Google Cloud environment.

Responsibilities

• Build and maintain end-to-end data pipelines using Python and Spark.
• Define workflows for scheduled and event-driven workload processing.
• Leverage GCP services to manage data movement and processing.
• Implement data processing tasks using Google Dataflow and Dataproc.

Required Skills

• 10+ years of experience in data engineering.
• Expertise in Google Cloud Platform (GCP).
• Hands-on experience with Google Dataflow and Dataproc.
• Proficiency with Cloud Composer, Pub/Sub, and BigQuery.
• Experience with Google Cloud Storage (GCS) and Cloud Functions.
• Strong programming skills in Python.
• Proficiency in SQL.
• Experience with Spark.
• Bachelor's degree in Computer Science or a related field.

Preferred Skills

• Experience with R.

Key Skills

Education

Bachelor's degree in Computer Science