Description

You will architect and design end-to-end data solutions on Azure, defining standards for ingestion, transformation, and storage. You own the implementation of scalable pipelines using Azure Data Factory and Databricks, optimizing Delta Lake architecture and medallion data layers.

This role is on-site.

Responsibilities

• Architect end-to-end data solutions using Azure Data Factory, Databricks, ADLS Gen2, and Azure Synapse.
• Define data architecture standards and best practices for batch and streaming ingestion, ETL/ELT transformation, and storage modeling.
• Lead implementation of scalable data pipelines using ADF and Databricks with PySpark and SQL.
• Design and optimize Delta Lake architecture, including Bronze/Silver/Gold medallion layers.
• Ensure data governance, security, and compliance through RBAC, masking, and encryption.

Required Skills

• 5+ years of experience with Azure Data Factory (pipelines, triggers, integration runtime).
• Strong proficiency in Azure Databricks, including PySpark, notebooks, and workflows.
• Expertise in Python, PySpark, and SQL for data processing and transformation.
• Experience with Delta Lake and lakehouse architecture patterns.
• Strong knowledge of data modeling, including star and snowflake schemas.
• Hands-on experience with Azure Data Lake Storage (ADLS Gen2) and Azure Synapse Analytics or SQL Server.
• Understanding of data ingestion patterns such as API, streaming, files, and CDC.

Preferred Skills

• Knowledge of CI/CD pipelines using Azure DevOps or GitHub.
• Experience with Infrastructure-as-Code tools like ARM or Terraform.
• Familiarity with monitoring tools such as Azure Monitor and Log Analytics.

Key Skills

Education

Bachelor's degree