Description

Build and maintain scalable data pipelines on AWS using Python, PySpark, and Databricks to support analytics workloads.

This role is on-site.

Responsibilities

• Develop and maintain ETL/ELT pipelines using Python and PySpark for large-scale data processing.
• Write and optimize SQL for data modeling, transformations, and analytics queries.
• Orchestrate data jobs using AWS Glue and AWS Databricks to ensure efficient execution and maintainable code.
• Design data ingestion and transformation workflows that load curated datasets into Amazon Redshift.
• Monitor pipeline performance, troubleshoot data issues, and improve reliability and processing efficiency.

Required Skills

• 5-10 years of relevant experience in data engineering.
• Proficiency in Python for data engineering tasks.
• Experience with PySpark for distributed data processing and transformations.
• Strong SQL skills for querying, data modeling, and performance optimization.
• Hands-on experience with AWS Glue, AWS Databricks, and Amazon Redshift.
• Knowledge of Hadoop ecosystem components.
• Bachelor's degree in Computer Science or a related field.

Key Skills

Education

Any Graduate