Description

You will build and maintain large-scale ETL pipelines using Python, PySpark, and AWS Glue, orchestrating workflows with AWS Step Functions and Lambda.

This role is on-site.

Responsibilities

• Develop ETL pipelines using Python and PySpark on AWS Glue.
• Orchestrate serverless workflows using AWS Step Functions and Lambda.
• Implement event-driven patterns using AWS SNS and SQS.
• Design and optimize data storage and querying in Amazon Redshift with performant SQL.
• Ensure data quality, monitoring, and error handling for operational support.

Required Skills

• 5+ years of experience with Python and PySpark for large-scale data processing.
• Hands-on experience with AWS services: Lambda, SNS, SQS, Glue, Redshift, Step Functions.
• Solid SQL skills with familiarity in data modeling and query optimization.
• Experience with ETL best practices, data quality checks, and monitoring/alerting.
• Familiarity with version control (Git) and basic DevOps/CI-CD workflows.
• Ability to collaborate with data consumers and stakeholders to translate requirements.

Preferred Skills

• Experience with infrastructure-as-code and documentation for reproducible deployments.

Key Skills

Education

Not specified