Description

Lead the architecture, design, and implementation of advanced analytics solutions using Azure Databricks and Fabric.

This role is on-site.

Responsibilities

• Architect and implement end-to-end data solutions, ensuring alignment with business requirements and best practices.
• Lead the development of data pipelines and ETL processes using Azure Databricks, PySpark, and Azure Data Factory.
• Integrate Azure Databricks with Azure Data Lake, Azure Synapse, and on-premise systems.
• Provide technical leadership and mentorship to the data engineering team, fostering continuous improvement.
• Ensure code quality, data security, scalability, and proper documentation of architecture and data flows.

Required Skills

• 8+ years of experience developing data ingestion and transformation pipelines using Databricks/Synapse notebooks and Azure Data Factory.
• Strong proficiency in Azure Databricks, including cluster management, notebook development, and Delta Lake.
• Expertise in big data technologies (Hadoop, Spark) and data processing frameworks (PySpark).
• Deep understanding of Azure services: Azure Data Lake, Azure Synapse, and Azure Data Factory.
• Experience with ETL/ELT processes, data warehousing, and building data lakes.
• Strong SQL skills and familiarity with NoSQL databases.
• Experience with CI/CD pipelines and version control systems like Git.
• Knowledge of cloud security best practices.

Preferred Skills

• Power BI for dashboarding, reporting, and optimizing query layers.
• Microsoft Fabric for analytics and integration tasks.
• Spark Streaming for processing real-time data streams and familiarity with Azure Resource Manager (ARM) templates.

Key Skills

Education

Any Graduate