Description

You will build and orchestrate autonomous AI agents, optimize LLM performance, and secure enterprise AI applications on the Databricks platform.

This role is remote.

Responsibilities

• Build agentic workflows and multi-agent systems using LangChain, CrewAI, AutoGen, or LlamaIndex.
• Deploy, fine-tune, and serve open-source LLMs (e.g., Llama 3) via Databricks Model Serving, optimizing latency and cost.
• Design advanced RAG pipelines leveraging vector search, embeddings, and enterprise data sources.
• Implement enterprise-grade security and governance using Unity Catalog for row/column-level security and lineage.
• Establish MLOps and AIOps pipelines including CI/CD, versioning, drift detection, and observability.

Required Skills

• 5+ years of experience in AI engineering or data science roles.
• Expertise in Databricks Lakehouse architecture, including Unity Catalog, Delta Lake, Vector Search, and Model Serving.
• Strong proficiency in Python for APIs, web apps, orchestration, and data processing.
• Hands-on experience with LLM frameworks such as LangChain, LlamaIndex, Semantic Kernel, or AutoGen.
• Experience with MLOps tools, specifically MLflow, CI/CD pipelines, and model lifecycle management.
• Solid foundation in data engineering with Spark, SQL, and large-scale data processing.
• Familiarity with streaming frameworks like Kafka or Structured Streaming.
• Knowledge of REST APIs and microservices architecture.

Preferred Skills

• Experience implementing LLM guardrails (e.g., NeMo Guardrails) for prompt injection prevention and hallucination mitigation.
• Background in building intuitive AI-driven applications using Databricks Apps (Streamlit/Dash) or modern web frameworks.

Key Skills

Education

Bachelor's degree