Description

You will design and maintain scalable data infrastructure on Azure using Databricks, focusing on ETL/ELT pipelines and lakehouse architectures.

This role is on-site.

Responsibilities

• Build and optimize ETL/ELT pipelines using Spark Structured Streaming and Delta Lake.
• Implement Medallion Architecture and manage data governance via Unity Catalog.
• Develop robust data solutions using Scala, PySpark, Python, and SQL.
• Deploy and manage infrastructure on Azure Cloud and AKS.

Required Skills

• 10+ years of experience in data engineering roles.
• Deep expertise with Azure Databricks and Delta Lake.
• Proficiency in Scala and PySpark for large-scale data processing.
• Strong command of SQL and Python for data manipulation and scripting.
• Experience with Unity Catalog for data governance.
• Knowledge of Spark Structured Streaming.
• Familiarity with Lakehouse and Medallion Architecture patterns.

Preferred Skills

• Experience with Azure Kubernetes Service (AKS).

Key Skills

Education

Any Gradute