Description

You will administer and optimize Azure Databricks platforms, ensuring stability, security, and performance for enterprise data workloads.

This role is on-site.

Responsibilities

• Configure and manage Azure Databricks clusters, including autoscaling, instance pools, and performance tuning.
• Implement Unity Catalog for fine-grained access controls, catalog-level security, and data governance.
• Automate deployments using Databricks CLI, REST APIs, and Databricks Asset Bundles.
• Build and maintain CI/CD pipelines with GitLab CI/CD, managing YAML definitions, runners, and environment promotions.
• Integrate Databricks with Azure infrastructure, including ADLS Gen2, Key Vault, and networking components like VNets.

Required Skills

• 4+ years administering Azure Databricks in enterprise environments.
• Strong proficiency with Terraform for modular, secure infrastructure-as-code.
• Hands-on experience with Spark-based workloads and data engineering patterns.
• Knowledge of Microsoft Azure fundamentals, including RBAC, managed identities, and Azure Monitor.
• Experience supporting compliance-driven environments, including audit evidence collection and change management.
• Familiarity with Git and version control best practices.
• 10+ years of total professional experience.

Preferred Skills

• Experience with MLOps, MLflow, and machine learning workload support on Azure Databricks.

Key Skills

Education

Any Graduate