Description

You will build and maintain scalable data pipelines and data products within the Quality Data Lake environment.

Responsibilities

• Design, develop, and implement scalable data pipelines using AWS native services.
• Build and optimize data products within the Quality Data Lake for stakeholder consumption.
• Implement and maintain ETL/ELT processes for ingesting diverse data sources.
• Design and execute testing strategies for data pipelines to ensure reliability.
• Monitor, troubleshoot, and optimize data lake components for performance and cost.

Required Skills

• 5+ years of proven experience as a Data Engineer focused on AWS cloud environments.
• In-depth knowledge of AWS native services: AWS Glue, Lambda, S3, Redshift, Kinesis, Athena.
• Strong proficiency in Python for data manipulation and automation.
• Expertise in ETL Processes and data modeling concepts.
• Proficiency in SQL and experience with database technologies.
• Experience implementing data governance and security best practices in AWS.
• Knowledge of data warehousing concepts.

Key Skills

Education

Any Graduate