Description

Design and maintain scalable data pipelines on Google Cloud Platform using PySpark, Airflow, and BigQuery.

This role is on-site.

Responsibilities

• Build and optimize ETL/ELT data pipelines using PySpark and Airflow.
• Implement data processing solutions on GCP, leveraging DataProc and Cloud Composer.
• Utilize BigQuery for data storage, querying, and performance optimization.
• Monitor pipeline reliability, troubleshoot issues, and ensure data quality and integrity.
• Collaborate with data scientists and analysts to translate requirements into technical solutions.

Required Skills

• 6+ years of experience in data engineering roles.
• Strong hands-on experience with PySpark, Python, and SQL.
• Proven expertise in building and maintaining data pipelines using Airflow.
• Experience working with GCP services: BigQuery, DataProc, and Cloud Composer.
• Strong understanding of data modeling, ETL/ELT processes, and data warehousing.
• Experience with performance tuning and optimization of large datasets.
• Familiarity with MySQL or other relational databases.

Preferred Skills

• Any Graduate

Key Skills

Education

Any Graduate