Description

Build and maintain scalable ETL pipelines and data workflows on AWS using Python and PySpark.

This role is on-site.

Responsibilities

• Develop ETL pipelines using Python and PySpark on AWS Glue and other compute platforms.
• Orchestrate serverless workflows with AWS Step Functions and Lambda.
• Implement event-driven patterns using AWS SNS and SQS for messaging.
• Design and optimize data storage in Amazon Redshift, writing performant SQL for transformations and reporting.
• Ensure data quality, monitoring, and operational support for pipelines while contributing to CI/CD and infrastructure-as-code.

Required Skills

• 5+ years of experience with Python and PySpark for large-scale data processing.
• Hands-on experience with AWS services: Lambda, SNS, SQS, Glue, Redshift, and Step Functions.
• Solid SQL skills with familiarity in data modeling and query optimization.
• Knowledge of ETL best practices, data quality checks, and monitoring/alerting.
• Familiarity with version control (Git) and basic DevOps/CI-CD workflows.

Preferred Skills

• Experience collaborating with data consumers and stakeholders to translate requirements into solutions.
• Background in contributing to reproducible deployments and documentation.

Key Skills

Education

Any Graduate