Description

Key Responsibilities
· Productionize Data Science and ML workflows developed in PySpark.
· Build and maintain production pipelines in Databricks.
· Develop CI/CD processes for deploying ML and AI solutions across development, QA, and production environments.
· Manage Databricks jobs, workflows, libraries, dependencies, configurations, and production releases.
· Implement MLflow or similar tooling for model tracking, versioning, and lifecycle management.
· Build monitoring and alerting for production ML pipelines.
· Troubleshoot pipeline failures, data issues, application integration issues, and  other production problems.
· Implement automated data quality and output validation checks.
· Help establish better testing practices for ML code, including unit, integration, and regression testing.
· Partner with Data Engineering teams to manage upstream data dependencies and downstream output requirements.
· Support production LLM / GenAI workflows, including API-based and batch inference processes.
· Monitor production performance, API usage, failures, and costs where applicable.
· Improve documentation and develop runbooks for recurring production support activities.
· Identify manual processes that can be automated or standardized.
· Help establish MLOps standards and reusable frameworks that can be applied across our portfolio of projects.

Experience / Skills Required
· 4+ years of experience in MLOps, ML Engineering, Data Engineering, DevOps, or a related engineering role.
· Strong Python skills.
· Strong Databricks and PySpark experience.
· Strong SQL skills.
· Experience building and supporting production data or ML pipelines.
· Experience with Git/GitHub and standard software development practices.
· Experience developing CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, or similar tools.
· Experience with production monitoring, logging, alerting, and troubleshooting.
· Experience working in a cloud environment such as Azure, AWS, or GCP.

Preferred
· Experience with MLflow and model lifecycle management.
· Experience supporting recommendation engines or other production ML applications.
· Experience with LLM / GenAI applications and API-based inference.
· Experience with Databricks Workflows, Unity Catalog, and Databricks deployment tooling.
· Experience with Docker and containerized applications.
· Experience implementing automated data quality or model monitoring.
· Familiarity with model drift, model performance monitoring, and ML regression testing

Key Skills

Education

Bachelor's degree