Description

You will design, build, and maintain scalable data pipelines and ETL solutions using Python/PySpark and AWS managed services.

Responsibilities

• Build and maintain ETL pipelines using Python and PySpark on AWS Glue and other compute platforms.
• Orchestrate workflows with AWS Step Functions and serverless components (Lambda).
• Implement messaging and event-driven patterns using AWS SNS and SQS.
• Design and optimize data storage and querying in Amazon Redshift.
• Ensure data quality, monitoring, error handling, and operational support for pipelines.

Required Skills

• 10+ years of experience in data engineering.
• Strong experience with Python and PySpark for large-scale data processing.
• Proven hands-on experience with AWS services: Lambda, SNS, SQS, Glue, Redshift, Step Functions.
• Solid SQL skills and familiarity with data modeling and query optimization.
• Experience with ETL best practices, data quality checks, and monitoring/alerting.
• Experience with SageMaker and EMR.
• Familiarity with version control (Git) and basic DevOps/CI/CD workflows.

Key Skills

Education

Any Graduate