Description

You will design, build, and maintain ETL/ELT pipelines within the AWS ecosystem.

Responsibilities

• Build and maintain ETL/ELT pipelines using AWS Glue and Apache Airflow.
• Create Apache Airflow DAGs to manage scheduling, dependencies, and error handling.
• Develop SQL scripts and optimize data models in Amazon Redshift.
• Implement data quality checks, schema validation, and monitoring for all pipelines.
• Integrate AWS services including S3, Lambda, Step Functions, Athena, and CloudWatch.

Required Skills

• 5+ years of experience in data engineering or related roles.
• Hands-on experience with AWS Glue, including Glue Jobs (PySpark) and Glue Catalog.
• Proficiency in Apache Airflow for workflow orchestration.
• Strong SQL skills for data modeling and script development.
• Experience with Amazon Redshift and Amazon Athena.
• Working knowledge of AWS S3 and AWS Lambda.
• Experience managing data workflows and job dependencies.
• Degree in any field of study.

Key Skills

Education

Any Graduate