Description

Design and implement ETL pipelines and metadata-driven Lakehouse architectures using Microsoft Fabric and Azure Data Factory.

This role is on-site.

Responsibilities

• Build and maintain metadata-driven Lakehouse architectures with threaded datasets to support multiple consumption patterns.
• Develop agent-specific data lakes and orchestration layers for AI agents querying across systems.
• Enable interactive data consumption via Power BI, Azure OpenAI, and other analytics tools.
• Ensure data quality, lineage, and governance across all ingestion and transformation processes.
• Optimize performance and cost across storage and compute layers while collaborating with product teams.

Required Skills

• 7+ years of experience in data engineering with a focus on Microsoft Azure Data stack.
• Strong expertise in Microsoft Fabric (Lakehouse, Dataflows Gen2, Pipelines, Notebooks).
• Proficiency in Azure Data Factory, Azure SQL, and Azure Data Lake Storage Gen2.
• Experience with Power BI and other visualization tools.
• Strong SQL, Python, and PySpark/Scala skills.
• Proven ability to build metadata-driven architectures and reusable components.
• Experience working with structured and semi-structured data (JSON, XML, CSV, Parquet).

Preferred Skills

• Familiarity with agent-based architectures and conversational AI integration.
• Strong understanding of data modeling and security best practices.

Key Skills

Education

Any Graduate