Description

Build and maintain data processing pipelines and cloud infrastructure on AWS.

This role is on-site.

Responsibilities

• Develop and optimize data processing pipelines using Python and PySpark for batch and event-driven workloads.
• Build AWS-based data workflows using S3, Glue, Lambda, Step Functions, and EventBridge.
• Provision and manage cloud infrastructure with Terraform, ensuring repeatable environments and secure configurations.
• Implement CI/CD pipelines with Git to automate build, test, and deployment for data services.
• Deploy and run workloads on ECS and EKS, ensuring reliability, observability, and performance.

Required Skills

• 5-10 years of total experience, with 4+ years in data engineering.
• Proficiency in Python and PySpark for data engineering development.
• Hands-on experience with AWS services: Lambda, S3, Glue, Step Functions, EventBridge, ECS, and EKS.
• Infrastructure as Code using Terraform and automation of environment provisioning.
• Experience with CI/CD pipelines, Git, and automated testing practices.
• SQL and RESTful API understanding.

Preferred Skills

• PowerShell scripting for operational automation.
• Code quality and security tooling such as SonarQube and Veracode.
• Automated testing tools including Pytest.

Key Skills

Education

Any Graduate