Description

Design and implement scalable data processing solutions using the Databricks Unified Analytics Platform.

Responsibilities

• Design efficient data processing solutions to meet business requirements.
• Integrate and transform data from multiple sources while ensuring quality and consistency.
• Build and implement data pipelines for efficient data movement and transformation.
• Write and optimize Apache Spark code for complex data processing logic.
• Collaborate with data scientists, analysts, and stakeholders to define data needs.

Required Skills

• 12-15+ years of experience building data strategies using various data management tools.
• 8-10 years of extensive experience with Databricks.
• Proficiency in Scala or Python for implementing data processing logic.
• Strong SQL skills for querying and manipulating data across various storage systems.
• Experience writing and optimizing Apache Spark code.
• Deep understanding of the Databricks Unified Analytics Platform and its components.
• Proficiency using Databricks Notebooks for interactive data exploration.
• Practical knowledge of ETL tools and methodologies.

Preferred Skills

• Bachelor's degree in Computer Science or related field.

Key Skills

Education

Bachelor's degree