Description

About the Role

We are looking for an experienced Lead Data Engineer to join our growing data practice in Bangalore. In this role, you will lead the architecture, design, and execution of enterprise-grade, cloud-native data pipelines and lakehouse solutions leveraging Azure Data Services and Azure Databricks.

Key Responsibilities

• Technical Leadership & Architecture: Lead the design and implementation of modern data platform architectures (Medallion Architecture: Bronze, Silver, Gold layers) using Azure Data Services and Databricks Delta Lake.
• Pipeline Engineering: Build and optimize end-to-end scalable, resilient batch and streaming ETL/ELT pipelines using PySpark, SQL, Azure Data Factory (ADF), and Databricks Workflows.
• Performance Tuning & Cost Optimization: Perform deep-dive Spark tuning, cluster policy optimization, partitioning, liquid clustering, and query optimization to ensure cost-efficiency and high execution speeds

Key Skills

Education

Any Graduate