Description

You will design and maintain scalable ETL/ELT data pipelines on AWS to support analytics and downstream consumption.

Responsibilities

• Build and optimize data processing workflows using Python and PySpark, focusing on transformations and performance tuning.
• Implement AWS cloud data services to orchestrate data movement, processing, and storage.
• Ensure pipeline reliability through monitoring, error handling, and troubleshooting production jobs.
• Translate stakeholder requirements into scalable pipeline designs and data models.
• Provision infrastructure and manage environments using Terraform for deployment consistency.

Required Skills

• 5+ years of experience building ETL/ELT pipelines.
• Hands-on expertise in Python for data processing and pipeline development.
• Proficiency in PySpark for large-scale data transformation.
• Strong understanding of AWS ecosystem (S3, Glue, Lambda, Redshift, EMR).
• Experience with data pipeline orchestration, monitoring, and troubleshooting.
• Familiarity with data modeling, validation, and performance optimization.
• Understand distributed data processing and big data concepts.

Preferred Skills

• Knowledge of Terraform for infrastructure as code and environment provisioning.

Key Skills

Education

Any Graduate