Description

You will lead the design, build, and optimization of data pipelines within the Azure ecosystem.

This role is on-site.

Responsibilities

• Develop and optimize data processing workflows using Databricks and PySpark.
• Design and implement data ingestion and transformation processes using Azure Data Factory (ADF).
• Manage and govern data assets within Azure Data Lake Storage Gen 2 (ADLS Gen 2).
• Develop and maintain data schemas and services in Azure SQL.
• Build and maintain Python packages for data engineering tasks.

Required Skills

• 5+ years of experience in the Azure Cloud platform.
• 4+ years of experience with Azure Databricks and PySpark.
• 2+ years of experience with Databricks workflow and Unity Catalog.
• 3+ years of experience using Azure Data Factory (ADF).
• 3+ years of experience with ADLS Gen 2.
• 3+ years of experience with Azure SQL.
• 2+ years of experience in Python programming and package builds.

Preferred Skills

• Experience with data governance and schema management in enterprise environments.
• Background in building reusable Python libraries for data engineering.

Key Skills

Education

Any Gradute