Description

You will build, scale, and monitor a complex BigData platform using Databricks, Snowflake, and Elasticsearch.

Responsibilities

• Design and implement the data platform, data science workbench, and ML hosting technology strategies in the cloud.
• Manage Databricks workspaces, including Unity Catalog, audit logs, user provisioning, SSO, and access control.
• Configure and install libraries, manage cluster and job policies, templates, and pools.
• Develop automation scripts using Terraform, YAML, Helm charts, az cli, aws-cli, Powershell, and cloud APIs.
• Implement monitoring and observability solutions using CSP tools, Zabbix, and Datadog SaaS.

Required Skills

• 5+ years of experience in operations or systems administration, specifically on Linux.
• Hands-on Databricks administration experience, including workspace management and security setup.
• Expertise in Docker and cloud-based containerization technologies.
• Strong coding skills in SQL and Python (PySpark) for code optimization.
• Proficiency with Terraform, YAML, and CI/CD pipelines.
• Experience managing storage account access and BigData platform disaster recovery.
• Knowledge of APIs and automation via CLI tools.
• Ability to assist in network, data security, and IAM initiatives.

Preferred Skills

• Experience with Zabbix for monitoring.
• Experience optimizing BigData platform costs and performance.

Key Skills

Education

Any Graduate