Description

Design and optimize large-scale data pipelines and analytics solutions within the Microsoft Azure ecosystem.

This role is on-site.

Responsibilities

• Build and maintain ETL/ELT frameworks using Azure Databricks, PySpark, and Delta Lake.
• Develop automated workflows using Azure Data Factory (ADF) pipelines and Databricks jobs.
• Process structured, semi-structured, and unstructured data using Spark DataFrames and Delta Lake architecture.
• Manage and optimize Databricks clusters and workspaces for performance and cost efficiency.
• Design analytical data models (star/snowflake schemas) and enforce data governance standards.

Required Skills

• 10+ years of relevant experience in data engineering.
• Expertise in Azure Databricks, Spark, and PySpark.
• Hands-on experience with Azure Data Factory (ADF).
• Proficiency with Azure Data Lake Storage (ADLS).
• Deep understanding of Delta Lake (ACID transactions, time travel).
• Experience with Azure Key Vault for secrets handling.
• Strong data modeling skills (Star / Snowflake schemas).
• Knowledge of distributed data processing and cloud data architecture.

Preferred Skills

• Azure Synapse Analytics.
• Cost optimization and performance tuning.
• Data governance and security implementation.

Key Skills

Education

Any Graduate