Description

You will design and own the Azure Data Lake and cloud data platform architecture, focusing on Databricks and Spark-based solutions.

This role is on-site.

Responsibilities

• Architect and implement scalable data solutions using Azure Databricks, Apache Spark, and Delta Lake.
• Develop and optimize data pipelines using Python, Scala, and SQL within the Azure ecosystem.
• Build and manage ETL workflows with Azure Data Factory (ADF) and Azure DevOps for CI/CD.
• Ensure data governance, performance optimization, and reliable data storage using ADLS Gen2.
• Define and enforce data architecture standards for cloud data platforms.

Required Skills

• 12+ years of experience in Data Engineering or Data Architecture.
• Deep expertise with Azure Databricks and Apache Spark.
• Strong proficiency in Python and Scala for data processing.
• Hands-on experience with SQL, Delta Lake, and Azure Data Lake Storage (ADLS Gen2).
• Proficiency in Azure Data Factory (ADF) for pipeline orchestration.
• Experience with Azure DevOps and CI/CD practices.
• Bachelor's degree in a related field.

Preferred Skills

• Experience in Financial Services or Insurance domains.
• Familiarity with Azure Synapse, Event Hub, or Cosmos DB.
• Microsoft Azure or Databricks certifications.

Key Skills

Education

Bachelor's degree