Description

You will build and maintain event-driven data pipelines and ETL/ELT workflows on AWS using Python and PySpark.

This role is on-site.

Responsibilities

• Build event-driven pipelines using Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
• Develop scalable ETL/ELT workflows with Python and PySpark, optimizing for performance and cost.
• Implement data quality standards, observability, and alerting via CloudWatch and Datadog.
• Establish data security practices including IAM, KMS encryption, and network governance.
• Create reusable frameworks, internal libraries, and CI/CD pipelines for automated deployments.

Required Skills

• 5+ years of experience in Python and PySpark for large-scale data processing.
• Deep experience building event-driven and streaming data pipelines.
• Strong SQL skills for analytical and ETL workloads.
• Hands-on experience with workflow orchestration tools such as Airflow or Step Functions.
• Proficiency with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation).
• Strong understanding of distributed systems, Spark performance tuning, and data modeling.
• Knowledge of cloud data security, encryption, networking, and compliance best practices.

Preferred Skills

• Experience working with Java.

Key Skills

Education

Any Graduate