Description

You will design and develop scalable ETL pipelines using AWS services.

Responsibilities

• Design and develop ETL pipelines leveraging AWS Glue, Lambda, S3, and Redshift/RDS.
• Integrate data from various sources including APIs, databases, and flat files into AWS platforms.
• Implement data transformation logic using PySpark, Python, or SQL within AWS services.
• Monitor, schedule, and orchestrate ETL workflows using AWS Step Functions, Glue Workflows, or Apache Airflow.
• Ensure data quality, consistency, and lineage using AWS Glue Data Catalog and Lake Formation.

Required Skills

• 5+ years of experience with ETL Processes.
• Proficiency in AWS services (Glue, Lambda, S3, Redshift/RDS, Step Functions).
• Strong skills in PySpark, Python, and SQL.
• Experience designing data lake architectures on AWS.
• Ability to optimize ETL performance and cost-efficiency.
• Knowledge of data security practices (IAM, encryption, VPC).
• Experience collaborating with data engineering and DevOps teams.

Key Skills

Education

Not specified