Description

You will design, develop, and maintain scalable data pipelines and ETL processes.

This role is on-site.

Responsibilities

• Develop and optimize data warehouses and data lakes using PySpark and AWS platforms.
• Implement data ingestion, transformation, and orchestration workflows using AWS Step Functions, Lambda, and Glue Jobs.
• Monitor, troubleshoot, and optimize pipeline performance using AWS CloudWatch.
• Collaborate with analysts and scientists to define data requirements and deliver reliable datasets.
• Automate data engineering tasks and contribute to reusable frameworks, adhering to CI/CD and IaC best practices.

Required Skills

• 12+ years of experience as a Data Engineer.
• Expertise in Python.
• Proficiency with PySpark.
• Strong experience with AWS services (Glue, Lambda, Step Functions).
• Hands-on experience with Databricks.
• Familiarity with CI/CD principles and tooling.
• Experience building and maintaining ETL processes.

Key Skills

Education

Any Gradute