Description

You will build and maintain the data and machine learning foundations for three web service applications.

Responsibilities

• Set up and manage Unity Catalog in Databricks to organize and secure data access across teams.
• Design and operationalize Feature Stores to support machine learning models in production.
• Build efficient data pipelines to process and serve features to ML workflows.
• Monitor and optimize the performance of pipelines and feature stores.
• Collaborate with teams to integrate data solutions using Databricks, Azure Cosmos DB, and other Azure tools.

Required Skills

• 5+ years of experience with Unity Catalog in Databricks for managing data assets and access control.
• 4+ years of hands-on experience with Databricks Feature Store or similar solutions.
• Proficiency in Java (Java 11).
• Proficiency in Python and Spark for data engineering tasks.
• Experience building and maintaining scalable ETL pipelines in Databricks.
• Familiarity with Azure tools including Azure Cosmos DB, ACR, and APIM.
• Familiarity with AKS for deploying and managing containers.
• Experience with monitoring tools such as Splunk or Datadog.
• Understanding of machine learning workflows and feature store integration.

Key Skills

Education

Any Gradute