Description

You will design and maintain scalable data pipelines and governance structures on the Azure cloud platform.

This role is on-site.

Responsibilities

• Build and deploy ETL/ELT pipelines using Azure Databricks, PySpark, and T-SQL.
• Create Delta Lake tables with ACID transactions and schema evolution for batch and real-time processing.
• Implement Unity Catalog for centralized data governance, access control, and lineage tracking.
• Analyze and resolve complex data issues end-to-end in real time.
• Write unit tests and apply SOLID principles to ensure data integrity and system cohesiveness.

Required Skills

• 7+ years of experience as a Lead Data Engineer.
• Expertise in Azure Databricks, PySpark, and T-SQL.
• Strong knowledge of Delta Lake and Azure SQL.
• Experience with Azure Data Factory and Azure Cloud infrastructure.
• Proficiency in Python and ETL/ELT development.
• Familiarity with Unity Catalog and ADLS Gen2.
• Experience with Microsoft TFS, Visual Studio, and DevOps practices.

Preferred Skills

• Healthcare domain knowledge.
• Experience with Scala and SSIS.

Key Skills

Education

Any Graduate