Description

You will manage and optimize the Databricks platform infrastructure on AWS.

Responsibilities

• Design, build, and maintain data infrastructure using Spark and Databricks SQL.
• Optimize Databricks workloads for performance, scalability, and cost efficiency.
• Implement and manage infrastructure using Terraform or CloudFormation.
• Support Data Scientists, Data Engineers, and Analysts in utilizing the platform effectively.
• Monitor infrastructure health and troubleshoot technical issues to ensure platform stability.

Required Skills

• 5+ years of hands-on experience with the Databricks platform, including Delta Lake and Spark.
• Proven experience building and maintaining cloud-based data infrastructure on AWS.
• Proficiency with Databricks SQL and data warehousing concepts.
• Experience with Infrastructure as Code (IaC) tools like Terraform or CloudFormation.
• Practical knowledge of data pipelines, ETL/ELT processes, and data modeling.
• Strong understanding of data lake and data engineering principles.
• Ability to implement security and governance best practices.
• Strong problem-solving and troubleshooting capabilities.

Preferred Skills

• Deep expertise with Terraform for infrastructure management.

Key Skills

Education

Any Gradute