Description

You will own the design, implementation, and management of scalable and secure Databricks platforms.

Responsibilities

• Design and implement Databricks workspaces and clusters, managing resource lifecycle and right-sizing.
• Automate infrastructure provisioning and deployments using IaC tools and scripting.
• Monitor platform health, performance, and cost, building necessary dashboards and integrating with monitoring tools.
• Troubleshoot cluster configuration issues, job failures, and performance bottlenecks within Databricks environments.
• Collaborate with data science and ML teams to deploy and maintain data pipelines on Databricks.

Required Skills

• 5+ years of experience architecting, deploying, and supporting Azure Databricks Platform and Azure Data Lake solutions.
• Hands-on experience with Azure cloud provider tools and architecture.
• Proficiency in scripting languages and CLI tools, including Python, Bash, PowerShell, Terraform, Ansible, and SQL.
• Experience implementing security and governance policies using Unity Catalog.
• Knowledge of ETL patterns and associated tools (e.g., Delta Lake, Snowflake Services).
• Experience working within an agile product development team.

Key Skills

Education

Any Gradute