Description

You will define and implement end-to-end architecture for data and ML platforms using Databricks.

This role is remote.

Responsibilities

• Design data pipelines and ETL workflows using Python and PySpark/Spark.
• Architect MLOps frameworks utilizing MLFlow, Databricks Asset Bundles, and Feature Store.
• Establish CI/CD pipelines and implement DevSecOps practices via Azure DevOps.
• Optimize Databricks clusters, job orchestration, and associated cost management.
• Ensure security, compliance, and governance across all data and ML workflows.

Required Skills

• 5+ years of experience in relevant architecture roles.
• Expertise in Databricks, including Workspace architecture, Asset Bundles, and Feature Store.
• Strong proficiency in Python and PySpark/Spark for data processing.
• Experience implementing MLOps practices, including MLFlow and model lifecycle management.
• Hands-on experience with Azure and Azure DevOps for CI/CD implementation.
• Knowledge of DevSecOps principles.
• Familiarity with distributed computing and data lakehouse architecture.
• Ability to perform performance tuning on data platforms.

Preferred Skills

• Any Graduate

Key Skills

Education

Any Gradute