Description

You will manage the deployment, configuration, and maintenance of Databricks workspaces and infrastructure.

Responsibilities

• Deploy and maintain Azure Databricks workspaces, including clusters, pools, and jobs.
• Manage security through IAM, RBAC, and Unity Catalog to enforce data governance and schema compliance.
• Optimize performance by monitoring cluster auto-scaling, fine-tuning Spark jobs, and using Databricks SQL for diagnostics.
• Automate infrastructure and cluster management using Terraform.
• Implement CI/CD pipelines for notebooks and integrate Databricks with Azure Data Factory, Synapse, Snowflake, and Kafka.

Required Skills

• 5+ years of experience in platform administration or data engineering.
• Databricks workspace management and runtime upgrades.
• Unity Catalog, catalog/schema creation, and grant management.
• Azure AD, Azure IAM, and SCIM for identity management.
• Spark job optimization and performance tuning.
• Terraform for infrastructure automation.
• Azure Data Factory, Synapse, Snowflake, and Kafka integration.
• GitHub or Azure DevOps for CI/CD pipelines.
• Delta Lake and Databricks Repos.

Key Skills

Education

Any Gradute