Description

You will build and maintain scalable data pipelines on Google Cloud Platform, optimizing data processing jobs and ensuring data quality.

This role is hybrid.

Responsibilities

• Develop and optimize data processing jobs using PySpark on GCP.
• Build and maintain batch and streaming pipelines using Dataflow and Dataproc.
• Orchestrate pipeline workflows using Cloud Composer (Airflow).
• Load and manage datasets in BigQuery, ensuring data quality and correctness.
• Manage data storage and movement using Cloud Storage and SQL-based transformations.

Required Skills

• 6+ years of relevant data engineering experience.
• Strong hands-on experience with PySpark and Python.
• Strong experience with Google BigQuery.
• Hands-on experience with Google Cloud Dataflow and Dataproc.
• Experience with Cloud Composer (Airflow) for workflow orchestration.
• Strong knowledge of Google Cloud Storage.
• Strong SQL skills with experience in data transformation and optimization.

Preferred Skills

• Experience in ETL/ELT pipeline development.
• Experience working on American Express (AMEX) projects or financial services domain.

Education

• Bachelor's or Master's Degree in Computer Science or related field.

Key Skills

Education

Any Graduate