Description

Responsibilities

• Design and build end-to-end AI agent workflows - from initial prompt design through production deployment
• Develop and optimize RAG pipelines - chunking strategies, embedding models, retrieval ranking, context window management - to ensure agents consistently pull the right information
• Build the prompt engineering layer - design, test, and systematically iterate on prompts and chain-of-thought strategies that produce accurate, consistent outputs across thousands of inputs
• Implement tool orchestration within agent workflows - connecting agents to databases, rule engines, validation systems, and formatting tools
• Build automated quality checks and validation layers that catch issues before outputs reach human reviewers
• Instrument solutions for measurement - work with the Data Scientist to build evaluation frameworks and ensure every solution can be tracked against its targets
• Deploy and maintain solutions in production
• Design and implement feedback loops that capture expert review data and translate it into measurable improvements in agent performance
• Build and maintain the data pipelines that feed each solution content pipelines for RAG, domain-specific databases (terminology, eligibility rules, validation data), and feedback collection systems
• Evaluate and integrate new AI models as they become available benchmark against current solutions for quality, cost, and speed
• Implement cost optimization strategies model routing, response caching, prompt optimization to reduce per-transaction AI spend as solutions scale
• Explore and prototype multi-modal capabilities (voice, image, document understanding) as the team’s scope expands beyond text-based workflows

Requirements

• 6+ years of software engineering experience, with at least 2–3 years focused on building LLM-based applications, RAG systems, or AI agent workflows
• Strong hands-on prompt engineering skills you know how to systematically test, evaluate, and iterate on prompt strategies across diverse inputs, not just write one-off prompts
• Deep understanding of RAG architectures embedding models, vector stores, retrieval strategies, re-ranking, and the tradeoffs involved in each design choice
• Experience building multi-step agent workflows with tool use, branching logic, and robust error handling
• Proficiency in Python and familiarity with LLM frameworks (LangChain, LlamaIndex, or similar)
• Comfort working in production environments you care about reliability, monitoring, and edge cases, not just happy-path demos
• Experience building and maintaining data pipelines you can set up the plumbing that feeds your AI systems, not just consume data someone else prepared
• Ability to work directly with non-technical domain experts to understand requirements, validate outputs, and iterate based on their feedback
• Experience with agile methodologies (Scrum, Kanban, or similar)

Key Skills

Education

Not specified