Description

Key Responsibilities

· Architect and design end-to-end data solutions on Azure using:
o Azure Data Factory (ADF)
o Azure Databricks
o Azure Data Lake (ADLS Gen2)
o Azure Synapse / SQL Database
· Define data architecture standards, frameworks, and best practices for:
o Data ingestion (batch & streaming)
o Data transformation (ETL/ELT)
o Data storage and modeling
· Lead the implementation of scalable data pipelines using ADF and Databricks (PySpark, SQL)
· Design and optimize Delta Lake architecture and medallion (Bronze/Silver/Gold) data layers
· Collaborate with stakeholders to understand business requirements and translate into technical solutions
· Ensure data governance, security, and compliance (RBAC, data masking, encryption)
· Provide technical leadership and mentorship to data engineering teams
· Optimize performance and cost of Azure data services
· Integrate real-time and batch processing pipelines

---

Required Skills & Qualifications

Technical Skills

· Strong experience with:
o Azure Data Factory (ADF) (pipelines, triggers, integration runtime)
o Azure Databricks (PySpark, notebooks, workflows)
o Delta Lake and lakehouse architecture

· Expertise in:
o Python / PySpark
o SQL
o Data modeling (star/snowflake schemas)

· Experience with:
o Azure Data Lake Storage (ADLS Gen2)
o Azure Synapse Analytics or SQL Server
o Data ingestion patterns (API, streaming, files, CDC)

· Knowledge of:
o CI/CD pipelines (Azure DevOps, GitHub)
o Infrastructure-as-Code (ARM, Terraform – preferred)
o Monitoring (Azure Monitor, Log Analytics)
 

Key Skills

Education

Bachelor's degree