Description

You will own the architecture, design, and implementation of advanced analytics solutions using Azure Databricks and Fabric.

Responsibilities

• Design and implement advanced analytics solutions using Azure Databricks and Fabric.
• Lead the development of data pipelines and ETL processes using Azure Databricks and PySpark.
• Integrate Azure Databricks with Azure Data Lake, Synapse, Data Factory, and on-premise systems.
• Oversee end-to-end data solution implementation, ensuring alignment with business needs.
• Provide technical leadership and mentorship to the data engineering team.

Required Skills

• 8+ years developing data ingestion and transformation pipelines using Databricks/Synapse notebooks and Azure Data Factory.
• Strong experience with Azure Databricks, including cluster management and Delta Lake.
• Proficiency in big data technologies (Hadoop, Spark) and data processing frameworks (PySpark).
• Deep understanding of Azure services: Azure Data Lake, Azure Synapse, and Azure Data Factory.
• Strong SQL skills and experience with ETL/ELT processes and data warehousing.
• Experience with CI/CD pipelines and Git version control.
• Solid understanding of Delta tables and Azure Data Lake Storage Gen2.
• Knowledge of cloud security best practices.

Key Skills

Education

Bachelor's degree