Description

You will design, build, and optimize scalable data pipelines on AWS to support analytics, reporting, and machine learning initiatives.

Responsibilities

• Develop and optimize ETL/ELT pipelines using PySpark, AWS Glue, and EMR.
• Automate ingestion from databases, APIs, and streaming sources.
• Write complex SQL queries for transformations, aggregations, and analysis.
• Build and manage data solutions on AWS (S3, Redshift, DynamoDB, Lambda).
• Implement monitoring, logging, and data quality checks for workflows.

Required Skills

• 6+ years of data engineering experience.
• Strong expertise in PySpark and SQL.
• Hands-on experience with AWS data services: Glue, EMR, Redshift, Athena, S3.
• Familiarity with Hadoop ecosystem.
• Experience working in agile and collaborative environments.
• Proficiency with Python scripting.
• Experience with AWS services including Lambda and DynamoDB.

Preferred Skills

• Exposure to streaming technologies (Kafka, Kinesis).
• Knowledge of data observability and data quality frameworks.

Key Skills

Education

Bachelor's degree