Description

You will own the design, implementation, and management of scalable and secure DataBricks platforms to ensure optimal performance.

Responsibilities

• Design, implement, and manage DataBricks workspaces and clusters, including resource right-sizing and lifecycle management.
• Automate infrastructure provisioning and deployment using Infrastructure-as-Code tools.
• Monitor platform health, performance, and cost by building dashboards and integrating with monitoring tools.
• Troubleshoot and resolve issues related to cluster configuration, job failures, and performance bottlenecks within DataBricks environments.
• Implement security and governance policies, including user management and access controls, using Unity Catalog.

Required Skills

• 5+ years of hands-on experience architecting, deploying, and managing Azure DataBricks Platform and Azure DataLake solutions.
• Required hands-on experience working with Azure cloud provider tools and architecture.
• Proficiency with scripting languages and CLI tools such as Python, Bash, PowerShell, Terraform, Ansible, SQL, and Azure CLI.
• Experience working in an agile product development team.
• Knowledge of DataBricks and Data Engineering principles.
• Ability to articulate system designs to various levels of leadership.

Key Skills

Education

Any Graduate