Description

You will design, implement, and optimize ETL/ELT pipelines and complex data models using PySpark and Snowflake to ensure high data quality and operational efficiency.

This role is on-site.

Responsibilities

• Design and implement robust ETL/ELT pipelines using PySpark and SQL to ingest data from APIs, flat files, and relational databases.
• Develop and manage complex data models, including Star/Snowflake schemas, and maintain Fact and Dimension tables within Snowflake.
• Monitor and tune Snowflake queries and Spark jobs to optimize performance, reduce latency, and manage computational costs.
• Implement automated data validation frameworks and testing procedures to ensure a single source of truth and high data reliability.
• Troubleshoot production pipeline issues and manage version control via Git.

Required Skills

• 6+ years of experience in data engineering.
• Strong proficiency with Snowflake and PySpark.
• Expertise in SQL and ETL/ELT pipeline development.
• Experience with data modeling, specifically Star and Snowflake schemas.
• Knowledge of data validation frameworks.
• Proficiency with Git for version control.

Preferred Skills

• Experience with Any Graduate degree equivalent.

Key Skills

Education

Any Graduate