Description

You will build and manage data pipelines using Python and PySpark.

Responsibilities

• Develop and maintain data processing workflows using AWS Glue and PySpark.
• Design and implement data models, including Fact and Dimension tables.
• Manage Slowly Changing Dimensions (SCD2) within the data architecture.
• Write and optimize complex SQL queries for data manipulation.

Required Skills

• 8+ years of experience in data engineering roles.
• Proficiency in Python programming.
• Advanced SQL skills.
• Experience with AWS Glue.
• Hands-on experience with PySpark.
• Expertise in data modeling, specifically Fact and Dimension tables.
• Experience implementing SCD2 (Slowly Changing Dimensions Type 2).
• Bachelor's degree or equivalent experience.

Key Skills

Education

Any Graduate