Description

You will define and implement end-to-end architecture for data and ML platforms using Databricks.

This role is remote.

Responsibilities

• Design data pipelines and ETL workflows leveraging Python and PySpark/Spark.
• Architect MLOps frameworks using MLFlow, Databricks Asset Bundles, and Feature Store.
• Establish CI/CD pipelines and DevSecOps practices using Azure DevOps.
• Optimize Databricks clusters, job orchestration, and cost management.
• Ensure security, compliance, and governance across all data and ML workflows.

Required Skills

• 7+ years of relevant experience.
• Expertise in Databricks Workspace architecture, Asset Bundles, and Feature Store.
• Strong proficiency in Python and PySpark/Spark.
• Experience with MLOps practices and MLFlow for model lifecycle management.
• Familiarity with Azure and Azure DevOps for CI/CD and DevSecOps.
• Knowledge of distributed computing and data lakehouse architecture.
• Experience with performance tuning in large-scale data environments.

Key Skills

Education

Any Gradute