Description

Design, build, and maintain scalable data pipelines and ETL processes on AWS.

This role is hybrid.

Responsibilities

• Develop and optimize data lakes and warehouses using PySpark and AWS Glue.
• Implement data ingestion, transformation, and orchestration with AWS Step Functions and Lambda.
• Monitor pipeline performance and troubleshoot issues using AWS CloudWatch.
• Automate tasks and build reusable data frameworks following CI/CD, Git, and IaC best practices.
• Ensure data quality, governance, and security across all data assets.

Required Skills

• 10+ years of professional experience in data engineering.
• Strong proficiency in Python for data processing and scripting.
• Hands-on experience with Databricks and PySpark for large-scale data processing.
• Deep knowledge of AWS services, specifically AWS Glue, Step Functions, Lambda, and CloudWatch.
• Experience implementing CI/CD pipelines and Infrastructure as Code (IaC).
• Proficiency with Git for version control.
• Any Graduate degree.

Preferred Skills

• LinkedIn profile created before 2020.
• Experience collaborating with data scientists and business stakeholders to define data requirements.

Key Skills

Education

Any Graduate