Description

Design and manage Databricks architecture and production-grade cluster deployments.

Responsibilities

• Deploy and configure Databricks clusters, SQL warehouses, and workspace settings.
• Manage cluster access, SQL warehouse permissions, and workspace asset security.
• Implement data governance, security protocols, and secure access to external storage.
• Monitor Databricks environments and develop custom plugins for monitoring statistics.
• Automate cloud operations and manage continuous integration and delivery pipelines.

Required Skills

• Hands-on experience with Databricks architecture and production cluster deployment.
• Deep knowledge of Delta Lake and its underlying structure.
• Proficiency in Python, SQL, Hive, Shell scripting, or Perl.
• Strong UNIX skills including user management, security, and permissions.
• Experience with cloud platforms including AWS, Azure, or GCP.
• Ability to set up VMs and manage hardware configurations (CPU, memory, disk, network, firewalls).
• Practical understanding of clustered deployment architecture and Databricks networking.
• Experience with CI/CD processes and cloud automation.

Preferred Skills

• Any graduate degree.

Key Skills

Education

ANY GRADUATE