Description

You will design and execute testing strategies for applications built around Retrieval-Augmented Generation (RAG) models and AI-driven systems.

This role is remote.

Responsibilities

• Design, execute, and automate functional, integration, regression, and performance tests for AI requirements.
• Validate AI model accuracy, retrieval quality, and system performance using metrics like precision, recall, and latency.
• Perform root-cause analysis on defects related to AI model predictions, retrieval accuracy, and user experience.
• Collaborate with engineering, data science, and product teams to define acceptance criteria and validate AI-driven functionalities.
• Implement automation tools and frameworks for continuous testing and monitor AI-specific quality metrics.

Required Skills

• 3+ years of experience in a Quality Engineering or QE role.
• Hands-on experience with test automation frameworks including Selenium, Cypress, JUnit, and TestNG.
• Experience automating tests for APIs and web interfaces using Postman.
• Familiarity with AI/ML systems, NLP, embeddings, and vector-based retrieval systems.
• Knowledge of AI model evaluation metrics such as precision, recall, F1-score, accuracy, BLEU, and ROUGE.
• Experience with performance and load testing tools like JMeter or Gatling.
• Familiarity with vector databases and embedding-based retrieval technologies like Pinecone, Weaviate, or Chroma.
• Strong analytical, troubleshooting, and root-cause analysis skills.
• Experience with CAPA metrics and methods.

Preferred Skills

• Experience testing Retrieval-Augmented Generation (RAG) or similar model architectures.
• Knowledge of Scrum and Kanban methodologies.

Key Skills

Education

Any Gradute