Description

You will own the end-to-end data engineering lifecycle, designing large-scale migration pipelines from on-prem systems to Azure Data Lake Storage and Azure Data Factory.

This role is on-site.

Responsibilities

• Build and maintain scalable data pipelines using Azure Databricks and Scala.
• Implement infrastructure-as-code for pipeline management using Terraform.
• Develop and optimize data workflows in Azure Data Factory for ingestion and transformation.
• Ensure data quality, logging, and traceability for audit and compliance.

Required Skills

• 8-10 years of experience in data engineering or data warehouse development.
• Hands-on expertise with Azure Databricks and Delta Lake.
• Strong proficiency in Scala for data processing and UI development within pipelines.
• Experience building and managing pipelines with Azure Data Factory.
• Proficiency with Terraform for infrastructure provisioning.
• Deep understanding of SQL, including complex joins, duplicate record identification, and data analysis.
• Experience working in agile development environments and sprint delivery.

Preferred Skills

• Any Graduate

Key Skills

Education

Any Graduate