Description

Design and manage production-grade Databricks environments and data architectures.

Responsibilities

• Deploy and configure production-grade Databricks clusters and SQL warehouses.
• Manage workspace settings, access controls, and security for workspace assets and external storage.
• Implement data governance and security protocols to secure data and external systems.
• Monitor Databricks performance and develop custom plugins for monitoring statistics.
• Automate cloud operations and manage CI/CD pipelines for data workflows.

Required Skills

• Extensive experience in Databricks architecture and clustered deployment.
• Deep knowledge of Delta Lake structure and Databricks networking.
• Proficiency in Python, SQL, Shell scripting, Perl, and Hive.
• Strong UNIX skills including user management, security, and permissions.
• Hands-on experience setting up VMs and configuring hardware resources like CPU, memory, and disk.
• Experience with cloud platforms including Azure, AWS, or GCP.
• Ability to manage connectivity and integration between Databricks and external systems.
• Understanding of firewalls and network security in a cloud environment.

Preferred Skills

• Any graduate degree.

Key Skills

Education

ANY GRADUATE