Description

You will manage and optimize Azure Databricks environments, ensuring high availability and performance for data engineering workflows.

Responsibilities

• Configure and maintain Azure Databricks clusters, workspaces, and access policies.
• Monitor cluster performance, manage costs, and implement autoscaling strategies.
• Support data engineers in optimizing Spark jobs written in Python, Scala, Java, and R.
• Implement data modeling best practices and ensure SQL query efficiency.
• Troubleshoot integration issues between Azure services and Databricks.

Required Skills

• 5+ years of experience in data engineering or platform administration.
• Strong proficiency in Python, Scala, Java, and R for data processing.
• Expertise in SQL and data modeling concepts.
• Hands-on experience with Azure Databricks administration and lifecycle management.
• Understanding of Spark architecture and optimization techniques.
• Ability to work effectively in a remote US-based team.

Key Skills

Education

Any Graduate