Description

You will design and optimize large-scale data pipelines using PySpark and Python.

This role is hybrid.

Responsibilities

• Develop and optimize PySpark data processing jobs for batch pipelines.
• Build and maintain data workflows on AWS for ingestion and transformation.
• Implement data quality checks and performance tuning for Spark workloads.
• Design scalable data pipelines for efficient data processing.
• Support deployment, troubleshooting, and operational improvements.

Required Skills

• 6+ years of experience in data engineering and pipeline development.
• Strong proficiency in PySpark and Python.
• Hands-on experience with AWS cloud services.
• Experience building scalable data processing pipelines using Spark.
• Knowledge of ETL/ELT processes and data workflow development.

Preferred Skills

• Experience with Azure cloud services.

Key Skills

Education

Any Graduate