Description

You will build and maintain scalable data processing solutions within a medallion architecture.

This role is on-site.

Responsibilities

• Manage and optimize data pipelines across Landing, Bronze, Silver, and Gold layers using AWS S3.
• Develop data transformation workflows using Python and PySpark.
• Perform data cuts and variance analysis to support business decision-making.
• Optimize the performance and integration of PostgreSQL application databases.
• Monitor and improve data quality and accuracy across all architectural layers.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in Python and PySpark.
• Strong understanding of Object-Oriented Programming (OOP) principles.
• Deep knowledge of the AWS ecosystem, specifically S3 and IAM.
• Experience implementing medallion architecture for data organization.
• Expertise in data variance analysis and creating data cuts.
• Solid understanding of PostgreSQL and database management.
• Experience handling large-scale data processing in distributed environments.

Preferred Skills

• Experience with AWS Lambda, Glue, and Redshift.
• Prior experience designing ETL pipelines in cloud-based environments.

Key Skills

Education

Any Graduate