Description

You will design and implement scalable data engineering solutions within a Databricks and AWS environment.

Responsibilities

• Design, develop, and implement efficient data science and data engineering solutions using Databricks.
• Optimize and fine-tune existing Databricks workflows to improve system performance and reliability.
• Translate business requirements into technical solutions through collaboration with cross-functional teams.
• Evaluate new Databricks features and assess their impact on the existing data ecosystem.
• Provide technical guidance and mentorship to the Databricks community regarding architecture and best practices.

Required Skills

• 5+ years of experience with Python, Spark, Scala, and big data technologies.
• Proven experience as a Databricks Engineer focusing on Databricks-AWS implementations.
• Strong understanding of Databricks architecture and its integration with AWS services.
• Experience designing and implementing scalable data processing pipelines.
• Proficiency with AWS cloud-based data storage and processing technologies.
• Competency in Java.
• Ability to troubleshoot and resolve complex technical issues.

Preferred Skills

• Experience with streaming, specifically Kafka and Databricks integrations.
• Knowledge of integrations with Starburst Trino, Collibra, or Immuta.
• Prior experience in the pharmaceutical or healthcare industry.

Key Skills

Education

Not specified