Description

You will design and maintain scalable data pipelines and ETL solutions using Python, PySpark, and AWS managed services, including SageMaker.

This role is on-site.

Responsibilities

• Build and maintain ETL pipelines using Python and PySpark on AWS Glue and other compute platforms.
• Orchestrate workflows with AWS Step Functions and serverless components like Lambda.
• Implement messaging and event-driven patterns using AWS SNS and SQS.
• Design, optimize, and query data storage in Amazon Redshift.
• Ensure data quality, monitoring, error handling, and operational support for all pipelines.

Required Skills

• 10+ years of experience in data engineering.
• Strong proficiency in Python and PySpark for large-scale data processing.
• Hands-on experience with AWS Lambda, SNS, SQS, Glue, Redshift, and Step Functions.
• Solid SQL skills with familiarity in data modeling and query optimization.
• Experience implementing ETL best practices, data quality checks, and monitoring/alerting.
• Experience with SageMaker and EMR.
• Familiarity with Git and basic DevOps/CI/CD workflows.

Key Skills

Education

Any Graduate