Description

Design and implement end-to-end data pipelines using Azure services for ingestion, transformation, and delivery.

This role is on-site.

Responsibilities

• Build and manage ETL/ELT workflows using Azure Data Factory (ADF), including orchestration and scheduling.
• Develop scalable data processing solutions using Azure Databricks with Spark SQL and PySpark.
• Implement and optimize data storage on Azure Data Lake Storage (ADLS Gen2) and manage data lifecycle patterns.
• Design and maintain data warehouses and analytics-ready datasets using Azure Synapse Analytics and Azure SQL Database.

Required Skills

• 7-13 years of experience in data engineering, building and operating production data pipelines on Azure.
• Expertise in Azure Data Factory (ADF) for pipeline orchestration and ETL/ELT workflow development.
• Proficiency in Azure Databricks for scalable Spark-based transformations and data processing.
• Experience with Azure Data Lake Storage (ADLS Gen2) for secure, optimized data storage and access patterns.
• Strong SQL-based data modeling skills using Azure Synapse Analytics and Azure SQL Database.
• Implementation of Azure Key Vault for secrets management and secure integration patterns.

Preferred Skills

• Experience with Azure Monitor and Log Analytics for monitoring, troubleshooting, and operational visibility.
• Knowledge of Azure Functions for event-driven or lightweight processing components.

Key Skills

Education

Any Graduate