Description

Administer the organization’s Databricks platform on AWS, ensuring stability, security, and efficient resource management.

Responsibilities

• Provision and manage AWS infrastructure (S3, IAM roles, Instance Profiles, KMS Keys) using Terraform.
• Manage role-based access and identity policies via Unity Catalog, service principals, and Delta Sharing.
• Automate administrative tasks using Databricks APIs, notebooks, and SQL queries.
• Monitor system health through dashboards and alerts, performing regular cluster health checks and backup/recovery procedures.
• Enforce governance through workflow tags, cluster tags, and regular privilege reviews of users and resources.

Required Skills

• 5+ years of experience administering the Databricks platform.
• Experience provisioning AWS infrastructure using Terraform.
• Proficiency with Databricks APIs and testing tools like Postman.
• Strong SQL skills for creating administrative queries and monitoring dashboards.
• Experience managing source code with GitHub.
• Deep understanding of Databricks workspace, development environments, and Unity Catalog.
• Experience performing compliance audits and maintaining audit documentation.

Preferred Skills

• Knowledge of Python.
• Familiarity with Datadog.

Key Skills

Education

Any Graduate