Description

You will lead the architectural design and implementation of advanced analytics solutions within the Azure ecosystem.

Responsibilities

• Lead the design and implementation of end-to-end data solutions and analytics architectures.
• Oversee the development of data pipelines and ETL/ELT processes using Azure Databricks and PySpark.
• Integrate Azure Databricks with Azure Data Lake, Azure Synapse, and Azure Data Factory.
• Provide technical leadership and mentorship to the data engineering team.
• Ensure architecture documentation, security compliance, and governance standards are met.

Required Skills

• 8+ years of experience developing data ingestion and transformation pipelines.
• Expertise in Azure Databricks, including cluster management, notebook development, and Delta Lake.
• Deep proficiency with Spark, Delta tables, and Delta Lake.
• Hands-on experience with Azure Data Factory, Azure Synapse, and ADLS Gen2.
• Strong SQL skills and experience building query layers using Databricks SQL.
• Experience with Auto Loader and Delta Live Tables for data ingestion.
• Proficiency in CI/CD pipelines and version control using Git.
• Solid understanding of cloud architecture and cloud security best practices.

Preferred Skills

• Experience with Power BI for reporting and dashboarding.
• Knowledge of Microsoft Fabric and Spark Streaming for real-time processing.
• Familiarity with Azure Resource Manager (ARM) templates and IaC practices.

Key Skills

Education

Any Gradute