Description

Design, build, and maintain data pipelines using Azure Databricks and Azure Data Factory.

This role is remote.

Responsibilities

• Design and maintain data pipelines using Azure Databricks and Azure Data Factory.
• Ingest and transform data from multiple source systems into the Azure Data Lake.
• Manage Delta Tables to ensure reliable and efficient data processing.
• Implement data governance and access management using Databricks Unity Catalog.
• Perform unit and end-to-end testing for all data pipelines.

Required Skills

• 3+ years of hands-on Databricks experience with certification.
• 3+ years of experience building and supporting pipelines using Azure Data Factory.
• 5+ years of strong SQL development experience.
• Hands-on experience with Databricks Delta Tables and Unity Catalog.
• Understanding of Agile development processes.

Preferred Skills

• Experience with data governance frameworks beyond Unity Catalog.
• Background in large-scale data lake architectures.

Key Skills

Education

Any Graduate