Description

Key Responsibilities

• Design and execute testing strategies for AI/ML, GenAI, and LLM applications.
• Develop and maintain automated testing frameworks.
• Test AI-generated responses for accuracy, consistency, bias, and reliability.
• Perform functional, API, integration, and data validation testing.
• Test RAG pipelines, AI chatbots, conversational AI, and AI agents.
• Identify hallucinations, prompt vulnerabilities, model drift, and potential AI risks.
• Ensure compliance with Responsible AI principles.
• Collaborate with developers, product teams, and data science teams.
• Prepare test reports, documentation, and quality metrics.
• Support CI/CD pipelines and production monitoring.

Required Skills

• 3–5+ years of experience in QA / Quality Engineering.
• 3+ years of experience in Test Automation.
• 2+ years of experience in AI/ML, GenAI, or LLM testing.
• Strong programming knowledge in Python or Java.
• Experience with API Testing and Integration Testing.
• Experience developing automation test frameworks.
• Knowledge of AI Model Evaluation and Responsible AI.
• Experience testing RAG, chatbots, AI agents, and LLM applications.
• Knowledge of hallucination detection, prompt testing, bias testing, and adversarial testing.
• Experience with CI/CD, Agile, and SDLC.
• Knowledge of AI testing frameworks, MLOps, and AI governance.

Good to Have

• Healthcare domain experience (Payer, Provider, Pharmacy, Claims).
• Experience with Azure, AWS, or GCP.
• Knowledge of accessibility testing and audit/compliance processes.
• Experience mentoring QA engineers.
• Strong communication and analytical skills

Key Skills

Education

Bachelor's degree