Description

JD

Minimum 4+ years of hands‑on experience administering Azure Databricks platforms in an enterprise environment.

Proven experience configuring and managing Azure Databricks platform components, including:

Clusters: autoscaling configurations, instance pools, cluster policies, and performance optimization
Jobs and Workflows: scheduling, concurrency controls, retries, alerts, and notifications
Workspace assets: notebooks, Git repos, libraries, init scripts, and secret scopes
Strong experience with Unity Catalog and implementation of enterprise data governance models, including fine‑grained access controls and catalog‑level security.

Experience with Databricks automation and deployment tooling, including:

Databricks CLI
Databricks REST APIs
Databricks Asset Bundles (DAB)
Experience integrating Databricks with BI Tools/analytics platforms.

Solid understanding of data engineering patterns, Spark‑based workloads, and techniques for tuning performance and optimizing resource usage.

Working knowledge of Microsoft Azure fundamentals, including:

Identity and access management using RBAC, service principals, and managed identities
Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Key Vault, Azure Monitor, and Log Analytics
Azure networking concepts such as VNets, private endpoints, and DNS resolution
Proven expertise with Terraform, including:

Modular design patterns
Remote state management
Multi‑environment deployment strategies
Secure infrastructure‑as‑code practices
Strong experience building and maintaining CI/CD pipelines using GitLab CI/CD, including:

YAML‑based pipeline definitions
Runner configuration
Environment promotion strategies
Approval gates and controlled deployments
Demonstrated experience supporting compliance‑driven and regulated environments, including:

Audit evidence collection and documentation
Periodic access reviews
Change management and release controls
Nice to Have experience on MLOps, including:

Experience supporting machine learning workloads on Azure Databricks
Familiarity with MLflow for experiment tracking, model registry, and lifecycle management
Exposure to MLOps pipelines, including model training, validation, and deployment automation
Understanding of model governance, versioning, and promotion across environments

Key Skills

Education

Any Graduate