Description

You design and implement ETL/ELT pipelines using PySpark and SQL to ingest data from APIs, flat files, and relational databases. You develop complex data models, including Star/Snowflake schemas, and maintain Fact and Dimension tables in Snowflake. You monitor and tune Snowflake queries and Spark jobs to optimize performance, reduce latency, and manage computational costs.

Responsibilities

• Build and maintain ETL/ELT pipelines with PySpark and SQL.
• Design and manage data models (Star/Snowflake schemas) in Snowflake.
• Tune Snowflake queries and Spark jobs for performance and cost efficiency.
• Ingest data from diverse sources including APIs and flat files.
• Maintain Fact and Dimension tables within the data warehouse.

Required Skills

• 8+ years of experience in data engineering.
• Expertise in PySpark and SQL.
• Strong experience with Snowflake data warehousing.
• Proven track record designing ETL/ELT pipelines.
• Experience with data modeling (Star/Snowflake schemas).
• Ability to optimize query performance and manage costs.

Preferred Skills

• Experience with relational database ingestion.
• Background in API data integration.

Key Skills

Education

Any Gradute