Description

Validate Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) architectures to ensure high-quality model outputs and retrieval accuracy.

This role is on-site.

Responsibilities

• Execute system integration testing for new or enhanced applications following Agile methodologies.
• Test LLMs and RAG architectures using vector databases and embeddings to validate retrieval accuracy.
• Implement automated testing frameworks for Gen AI applications using tools like DeepEval or RAGAS.
• Develop custom validation scripts to evaluate model accuracy, relevancy, faithfulness, and performance benchmarks.
• Perform prompt engineering tests to optimize model responses and execute SQL queries in Snowflake to validate data integrity.

Required Skills

• 6+ years of experience in software testing and implementation lifecycles.
• Hands-on experience testing LLMs and RAG architectures.
• Proficiency in Python and AI testing libraries including LangChain, Pytest, and PyTorch.
• Experience with Selenium for automated application testing.
• Ability to write and execute SQL queries within Snowflake.
• Strong knowledge of Agile methodology and the software development lifecycle.
• Experience using Jira and ALM for issue and quality management.
• Proven ability to manage project milestones, dependencies, and status reporting.

Preferred Skills

• Experience with C programming.

Key Skills

Education

Any Graduate