Description

Key Skills: Azure ADF, Azure, Azure AD, Pyspark, Python

Roles and Responsibilities:

• Architect and implement data solutions leveraging Azure and the Databricks ecosystem.
• Own the complete lifecycle of data platform implementations--from requirements gathering and platform selection to architecture design and deployment.
• Work closely with data scientists, application developers, and business stakeholders to deliver enterprise-grade data solutions.
• Continuously explore emerging technologies to solve business problems creatively.
• Mentor and guide a team of data engineers, promoting best practices and innovation.
• Contribute to organizational initiatives including capability building, solution development, talent acquisition, and industry events.

Skills Required:

• 8+ years of overall IT experience with at least 4+ years of hands-on experience in Azure and Databricks.
• Proven experience leading at least 2 end-to-end Data Lakehouse implementations using Azure Databricks (Medallion Architecture).
• Deep expertise in Databricks ecosystem: PySpark, Notebooks, Unity Catalog, Delta Live Tables, Workflows, SQL Warehouse, Mosaic AI, AI/BI Genie.
• Strong experience building modern data platforms using Azure services such as Azure Databricks, Azure Data Factory, ADLS Gen2, SQL Database, Microsoft Fabric, Event Hub, Stream Analytics, Cosmos DB, Azure Purview, Log Analytics, and Azure Data Explorer.
• Experience designing and implementing metadata-driven data engineering frameworks.
• Strong programming and debugging skills in Python and SQL with performance tuning expertise.
• Solid understanding of data modeling (Dimensional & 3NF).
• Exposure to LLM/GenAI-based application development.
• Experience with CI/CD pipelines using Git, Jenkins, or Azure DevOps.
• Familiarity with big data ecosystems like Cloudera (CDH) or Hortonworks (HDP) is a plus.
• Exposure to tools like Neo4j, Elasticsearch, and vector databases is an added advantage.
• Understanding of Azure infrastructure, networking, security, and governance is a plus.
• Strong leadership, communication, and stakeholder management skills.

Education: Bachelor's degree (B.E/B.Tech) in Computer Science, Information Technology, or a related field from a reputed institute (preferred)

Key Skills

Education

Any Graduate