Description

You will manage and scale a complex Big Data platform utilizing Databricks, Snowflake DB, and ElasticSearch cloud.

Responsibilities

• Build, scale, and monitor Databricks-based data science and client hosting platforms.
• Design architecture and define the technology roadmap for the data platform and data science workbench.
• Implement monitoring, alerting, and observability solutions using CSP tools, Zabbix, and Datadog SaaS.
• Develop automation scripts using Terraform, YAML, Helm charts, az cli, aws-cli, Powershell, and cloud APIs.
• Manage Databricks workspace administration, including users, groups, SSO, provisioning, and access control.

Required Skills

• 5+ years of experience in operations or systems administration, specifically on Linux.
• Expertise in Databricks administration, including Unity Catalog, audit logs, and workspace management.
• Hands-on experience managing cluster and jobs, policies, templates, and pool configurations.
• Proficiency in SQL and Python (PySpark) with a focus on code optimization.
• Strong skills in Docker and cloud-based containerization technologies.
• Experience managing storage account access and implementing security, privacy, and IAM initiatives.
• Ability to troubleshoot network issues, Private Endpoints, and platform-level errors.
• Competency in Cloud Administration and automation via Powershell, Terraform, or CLI tools.

Preferred Skills

• Experience optimizing Databricks usage for both performance and cost efficiency.

Key Skills

Education

Any Graduate