Description

Architect and lead the delivery of enterprise-scale Azure Lakehouse platforms and data solutions for petabyte-scale processing.

This role is on-site.

Responsibilities

• Design real-time streaming solutions using Apache Kafka and Spark Structured Streaming for high-volume data processing.
• Lead migration of on-premise data warehouses to Azure Synapse and Databricks to improve performance and reduce costs.
• Build reusable PySpark frameworks for data ingestion, transformation, and validation across multiple projects.
• Implement end-to-end Azure Data Factory pipelines with CI/CD and parameter-driven orchestration.
• Manage and lead teams of 40+ members, governing delivery processes and driving innovation initiatives.

Required Skills

• 5+ years of experience with Azure Databricks, PySpark, and Apache Spark.
• Strong proficiency in SQL, Big Data concepts, Python, and Java.
• Experience with Azure Synapse and Apache Kafka.
• Hands-on experience with Azure Data Factory.
• Proven track record of managing large engineering teams (40+ members).
• Experience with architectural guidance and technical mentorship.

Preferred Skills

• Experience with Delta Lake, Unity Catalog, and Azure Data Lake Storage Gen2.
• Knowledge of CI/CD with Jenkins and GitHub, Dynatrace, and microservices.
• Capital markets and securities processing experience.

Key Skills

Education

Any Graduate