Description

You will design and maintain end-to-end data pipelines using Microsoft Fabric and Azure services, owning the architecture from ingestion to consumption.

Responsibilities

• Build and manage Lakehouse architecture using OneLake, Data Pipelines, Notebooks, and SQL Endpoints.
• Develop scalable ETL/ELT workflows using Python, PySpark, and SQL, including incremental loads, CDC, and watermarking logic.
• Implement data migration from on-premises systems to Azure, ensuring transformation, cleansing, validation, and reconciliation.
• Optimize SQL queries, Spark jobs, and data pipelines for performance while maintaining Bronze, Silver, Gold layers.
• Automate deployments using Azure DevOps CI/CD and monitor production pipelines for reliability.

Required Skills

• 5+ years of experience with Python, PySpark, and SQL.
• Strong proficiency in Microsoft Fabric, including OneLake and Data Pipelines.
• Experience with Azure Data Factory, ADLS Gen2, Synapse, and Databricks.
• Knowledge of Azure DevOps for CI/CD automation.
• Understanding of data governance, security, access controls, and lineage.
• Ability to support Power BI reporting and semantic models.
• Bachelor's degree in any discipline.

Key Skills

Education

Any Gradute