Description

Design and implement data pipelines on AWS to process large volumes of data from diverse sources.

Responsibilities

• Develop ETL workflows using Python and SQL for data processing.
• Execute data cleansing and validation processes to ensure quality.
• Integrate CI/CD pipelines to build artifacts and deploy changes to higher environments.
• Collaborate with cross-functional teams to deliver data solutions.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and SQL.
• Hands-on experience with AWS services: Glue, Glue Crawlers, Lambda, Redshift, Athena, S3, EC2, and IAM.
• Experience with AWS monitoring and logging using CloudWatch and setting up alerts.
• Knowledge of scheduling frameworks such as Airflow or AWS Step Functions.
• Practical experience with cloud deployment and CI/CD integration.
• Core competency in Data Warehousing and Databases.
• Strong data validation and cleansing capabilities.

Key Skills

Education

ANY GRADUATE