Description

Manage and scale production-grade Databricks and ADLS environments on Microsoft Azure for multiple tenant teams.

Responsibilities

• Set up and maintain production-scale Databricks environments and ADLS.
• Migrate Hadoop platform workloads to Databricks on Azure.
• Develop automated tools for self-service configuration, deployment speed, and monitoring.
• Implement security compliance and data lockdown solutions.
• Build CI/CD tooling and automate tenant onboarding processes.

Required Skills

• 10+ years of professional experience.
• 5+ years working with large datasets using Spark, Hadoop, and Kafka on major cloud vendors.
• 2+ years of hands-on experience with Databricks on Cloud.
• Experience managing Apache Hadoop ecosystems including Yarn, Hive, Oozie, and HDFS.
• Practical experience with Jenkins, Spinnaker, Salt, and Azure DevOps.
• Proficiency in Linux production troubleshooting.
• Experience migrating on-prem Hadoop workloads to Azure.
• Strong automation skills using scripting languages and tools.
• Experience implementing monitoring and alerting via Splunk, PagerDuty, or Prometheus.

Key Skills

Education

Any gradudate