Description

You will design and build scalable ETL pipelines leveraging core AWS services.

Responsibilities

• Design and develop ETL pipelines using AWS Glue, Lambda, S3, and RDS/Redshift.
• Integrate data from various sources, including APIs, databases, and flat files, into AWS data platforms.
• Implement data transformation logic using PySpark, Python, or SQL within AWS Glue or Lambda.
• Orchestrate ETL workflows using AWS Step Functions, Glue Workflows, or Apache Airflow on MWAA.
• Optimize ETL performance and cost, while ensuring data quality and lineage via Glue Data Catalog and Lake Formation.

Required Skills

• 5+ years of experience in ETL Pipelines development.
• Proficiency in PySpark, Python, and SQL.
• Deep experience with core AWS Services (Glue, S3, Lambda, Redshift/RDS).
• Experience managing data integration from diverse sources (APIs, databases).
• Ability to implement security practices using IAM roles and VPC configurations.
• Experience documenting data flows and architecture.

Key Skills

Education

Any Gradute