Lead Data Engineer

You will lead the architecture and implementation of an enterprise-grade data platform.

Responsibilities

• Design and implement a Medallion architecture (Bronze/Silver/Gold layers) for data management.
• Develop real-time data ingestion, validation, and transformation pipelines for diverse file formats.
• Build and orchestrate event-driven workflows using serverless Azure Functions, Event Hub, and Durable Functions.
• Integrate Fabric Lakehouses and Synapse Data Warehousing with transactional systems and UI components.
• Lead and mentor a team of data engineers, setting coding and documentation standards.

Required Skills

• 8-12 years of hands-on data engineering experience, including 3+ years in an architecture or leadership role.
• Strong expertise in Microsoft Azure, specifically Microsoft Fabric (Data Lakehouses, Pipelines, Real-Time Analytics, Synapse).
• Advanced knowledge of Python, PySpark, SQL, and workflow orchestration.
• Experience building event-driven, serverless architectures in production.
• Deep understanding of schema-agnostic ingestion, validation logic, and dynamic rule engines.
• Proficiency with Azure Data Factory, Azure Functions, Event Hub/Service Bus, and Delta Lake on ADLS Gen2.
• Experience enforcing RBAC, end-to-end encryption, and audit trails across data layers.
• Familiarity with Rest APIs and integrating with transactional UI components.
• Knowledge of data privacy, regulatory compliance, lineage, and auditability.