Description

You will design and implement scalable data pipelines on AWS to process large volumes of data from diverse sources.

This role is on-site.

Responsibilities

• Design and implement scalable data pipelines on AWS.
• Perform data cleansing and validation to ensure high data quality.
• Develop ETL processes using Python and SQL.
• Integrate CI/CD pipelines to build artifacts and deploy changes to higher environments.
• Manage scheduling and orchestration using Airflow or AWS Step Functions.

Required Skills

• 5+ years of experience in data engineering.
• Hands-on experience with AWS Glue, Glue Crawlers, and Lambda.
• Proficiency in SQL and Python for ETL development.
• Experience with AWS Redshift, Athena, and S3.
• Working knowledge of EC2 and IAM.
• Experience with monitoring and logging via AWS CloudWatch and alert setup.
• Practical knowledge of cloud deployment and CI/CD integration.

Preferred Skills

• Experience with Apache Kafka.

Key Skills

Education

Any Graduate