Description

This role requires a Lead Data Engineer to manage and build data pipelines within the Azure ecosystem.

Responsibilities

• Develop and optimize data processing workflows using Databricks and PySpark.
• Design and implement data ingestion and transformation processes using Azure Data Factory (ADF).
• Manage and govern data assets within Azure Data Lake Storage Gen 2 (ADLS Gen 2).
• Develop and maintain data schemas and services in Azure SQL.
• Build and maintain Python packages for data engineering tasks.

Required Skills

• 5+ years of experience in the Azure Cloud platform.
• 4+ years of experience with Azure Databricks and PySpark.
• 2+ years of experience with Databricks workflow and Unity Catalog.
• 3+ years of experience using Azure Data Factory (ADF).
• 3+ years of experience with ADLS Gen 2.
• 3+ years of experience with Azure SQL.
• 2+ years of experience in Python programming and package builds.

Key Skills

Education

Any Gradute