Description

Lead end-to-end data engineering projects focused on Databricks development and complex data migrations from legacy systems.

This role is on-site.

Responsibilities

• Design and develop data ingestion and transformation pipelines using Scala within Databricks.
• Execute data migrations from legacy SQL Server and Oracle systems to Databricks.
• Implement data warehousing solutions using ETL concepts and Slowly Changing Dimensions (SCD).
• Manage data lake architectures and Delta table implementations.
• Write complex queries and functions to handle data from multiple source systems.

Required Skills

• 5+ years of experience in data engineering.
• Extensive development experience in Databricks using Scala.
• Strong proficiency in Spark SQL.
• Deep understanding of data warehousing, ETL, and SCD types.
• Expertise in data lake and Delta table concepts.
• Experience with SQL Server or Oracle databases.
• Proficiency in writing complex queries using T-SQL or PL/SQL.
• Bachelor's degree in a relevant field.

Preferred Skills

• Experience with Snowflake.
• Background in Big Data Hadoop environments.

Key Skills

Education

Bachelor's degree