Description

Design and maintain scalable data Lakehouse solutions on Microsoft Fabric and Azure Databricks.

This role is on-site.

Responsibilities

• Design and implement data processing pipelines using Azure Databricks, Spark, and Microsoft Fabric.
• Write optimized Python (PySpark) and SQL code for large-scale data ingestion, transformation, and integration.
• Architect and maintain data governance, lineage, and cataloging using Microsoft Purview.
• Build and optimize ETL workflows across Azure services, ensuring data quality and security.
• Manage version control with Git and follow CI/CD best practices for data engineering.

Required Skills

• 5+ years of hands-on experience with Azure Databricks and Spark.
• Proficiency in Python (PySpark) and SQL for data processing.
• Experience with Microsoft Fabric for analytics and integration.
• Proven knowledge of Microsoft Purview for data governance and compliance.
• Strong understanding of data lake architecture and ETL processes.
• Experience with Azure Data Lake, Synapse, and Data Factory.
• Ability to optimize performance for scalability.

Preferred Skills

• Experience with data quality management and validation practices.
• Strong technical writing skills for documentation and design artifacts.

Key Skills

Education

Bachelor's degree