Description

You will manage and optimize the Databricks environment to ensure performance, scalability, and security compliance.

Responsibilities

• Configure and manage Databricks clusters to meet project requirements while optimizing for cost and performance.
• Design, implement, and troubleshoot scalable data processing pipelines.
• Monitor cluster health, implement logging solutions, and conduct regular maintenance and patches.
• Analyze consumption reports to identify and correct abnormal usage patterns.
• Document configurations, best practices, and provide technical support to internal teams.

Required Skills

• 5+ years of experience in a Databricks Engineer or similar role.
• Strong proficiency in Databricks and Apache Spark.
• Experience with cloud platforms including AWS, Azure, or GCP.
• Proficiency in Python, Scala, or Java programming.
• Strong SQL skills and understanding of data architecture and ETL processes.
• Experience with system integration and Databricks support.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Databricks Certification.
• Experience with big data technologies, distributed computing, and machine learning frameworks.

Key Skills

Education

Any Graduate