Description

Build and maintain event-driven data pipelines and data lake structures on AWS.

This role is on-site.

Responsibilities

• Develop ETL/ELT workflows using Python, PySpark, and AWS services like Glue, EMR, Lambda, and Step Functions.
• Design and manage data lake/warehouse structures using S3, Glue Catalog, Athena, and Redshift.
• Build streaming solutions with Kinesis/MSK, implementing checkpointing, stateful transformations, and idempotency.
• Implement data quality, observability, and security standards including IAM, KMS, and CloudWatch/Datadog.
• Create reusable frameworks, CI/CD pipelines, and mentor junior engineers.

Required Skills

• 8+ years of overall experience with 5+ years in Data Engineering.
• Strong expertise in Python and PySpark for large-scale data processing.
• Advanced hands-on experience with AWS (S3, Glue, EMR, Lambda, Kinesis/MSK, DynamoDB, Athena, Redshift).
• Deep experience building event-driven and streaming data pipelines.
• Strong SQL experience for analytical and ETL workloads.
• Experience with workflow orchestration tools such as Airflow or Step Functions.
• Hands-on experience with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation).

Preferred Skills

• Experience working with Java.
• Experience with Spring Boot.

Key Skills

Education

Any Graduate