Description

You will deploy and optimize open-source LLMs in on-premises environments, building secure RAG pipelines and delivering working prototypes to internal teams.

This role is on-site.

Responsibilities

• Deploy Meta Llama 3, Mistral, and Mixtral models in private, air-gapped, or on-prem settings.
• Build and tune Retrieval-Augmented Generation (RAG) pipelines using vector databases and CPU-based inference.
• Implement data privacy controls, audit logging, and access management for enterprise security.
• Create reference architectures and deliver knowledge transfer documentation.

Required Skills

• 5+ years of experience with Python for LLM inference and integration.
• Strong knowledge of prompt engineering and model quantization.
• Hands-on experience with vector databases: Qdrant, Chroma, Milvus, or pgvector.
• Understanding of data privacy, air-gapped deployments, and enterprise governance.
• Experience with inference frameworks such as vLLM, llama.cpp, or Hugging Face Transformers.

Preferred Skills

• Experience with LangChain or LlamaIndex.
• Familiarity with Rust, Go, or C++ for high-performance services.
• Docker and Kubernetes proficiency for on-prem deployments.

Key Skills

Education

Any Graduate