Description

You will manage and optimize Azure Databricks environments to ensure efficient data processing and platform stability.

Responsibilities

• Optimize Azure Databricks performance and manage complex workloads.
• Implement and manage data governance using Databricks Unity Catalog and Delta Lake.
• Automate infrastructure deployment using Terraform, Bicep, or ARM templates.
• Build and maintain CI/CD pipelines using Azure DevOps, GitHub Actions, or Jenkins.
• Enforce network security, identity management via AAD, and encryption best practices.

Required Skills

• 5+ years of experience in Azure Databricks administration and performance optimization.
• Proficiency in Apache Spark, PySpark, SQL, and Scala.
• Hands-on experience with Databricks Unity Catalog, Delta Lake, and MLflow.
• Working knowledge of Azure Data Lake, Azure Synapse, and Azure Key Vault.
• Experience with Infrastructure as Code (Terraform, Bicep, or ARM Templates).
• Experience with CI/CD automation (Azure DevOps, GitHub Actions, or Jenkins).
• Strong understanding of AAD identity management and network security.
• Ability to troubleshoot complex Databricks workloads.
• Any Graduate degree.

Key Skills

Education

Any Graduate