Description

You will build and maintain end-to-end data and ML solutions using Azure Databricks, Scala, and Python.

This role is remote.

Responsibilities

• Build and optimize data pipelines using Apache Spark, Structured Streaming, and Delta Lake.
• Develop and manage ML workflows, including model development, evaluation, and monitoring via MLflow.
• Implement CI/CD pipelines and version control practices for data and ML systems.
• Troubleshoot Spark applications and resolve complex data pipeline issues.
• Mentor team members and work independently on loosely defined problems.

Required Skills

• 5+ years of experience in data engineering.
• Strong proficiency in Azure Databricks and Delta Lake.
• Hands-on experience with Scala and Python.
• Deep knowledge of Apache Spark and Structured Streaming.
• Experience with Azure or AWS cloud environments.
• Familiarity with Git and CI/CD pipelines.
• Understanding of MLOps fundamentals, including versioning and monitoring.
• Experience handling unstructured data processing and transformation.

Preferred Skills

• Exposure to production ML or AI systems.
• Strong analytical and problem-solving skills.
• Mentoring experience.

Key Skills

Education

Any Graduate