Description

You design, develop, and optimize prompt chains and evaluation frameworks for enterprise Generative AI applications.

This role is on-site.

Responsibilities

• Design and refine prompts and prompt chains for summarization, classification, extraction, reasoning, and Q&A over enterprise data.
• Build and maintain evaluation frameworks covering accuracy, faithfulness, hallucination rate, bias, latency, and cost using automated metrics and human-in-the-loop review.
• Conduct systematic experimentation across models, prompting techniques, and retrieval strategies to drive token and cost optimization through prompt compression, caching, and model routing.
• Translate model outputs into clear insights and dashboards for business stakeholders.

Required Skills

• 3-5 years of experience in data science, machine learning, analytics, or applied AI.
• 1-2 years working hands-on with LLMs or Generative AI in production or near-production settings.
• Strong proficiency in Python, Machine Learning, and Data Science fundamentals.
• Expertise in Prompt Engineering and Generative AI application development.
• Experience with LLM Evaluation methodologies and metrics.
• Knowledge of RAG Pipelines and associated architectures.

Preferred Skills

• Experience with vector databases (Pinecone, Weaviate, pgvector) and embedding model selection.
• Familiarity with agentic frameworks, tool use, and multi-step workflows.
• Experience with observability tools for LLMs such as LangSmith, Langfuse, or Arize.

Key Skills

Education

Any Graduate