Description

You will lead a team of data engineers and define enterprise data engineering best practices for the Azure platform.

This role is on-site.

Responsibilities

• Lead and mentor a team of data engineers while driving architectural decisions for data platform modernization.
• Design and develop scalable data pipelines using Azure Data Factory (ADF) and integrate data from multiple sources.
• Build and optimize data transformations using Azure Databricks with PySpark, SQL, and Scala.
• Implement and maintain Lakehouse architecture using Delta Lake, ensuring data quality, lineage, and governance.
• Implement CI/CD pipelines and automate deployments using Infrastructure as Code tools.

Required Skills

• 5+ years of experience building large-scale distributed data pipelines.
• Strong hands-on experience with Azure Databricks (PySpark, Spark SQL) and Azure Data Factory (ADF).
• Expertise in Python and SQL for data engineering tasks.
• Experience with Delta Lake and Lakehouse architecture design.
• Strong knowledge of Azure Data Lake Storage (ADLS Gen2) and data warehousing concepts.
• Proficiency in ETL/ELT best practices and data modeling techniques (star/snowflake schemas).
• Experience with Azure DevOps, Git version control, and CI/CD pipelines.

Preferred Skills

• Familiarity with streaming frameworks such as Kafka or Event Hub.
• Experience with Azure Synapse Analytics and Infrastructure as Code (ARM/Bicep/Terraform).

Key Skills

Education

Bachelor's degree