Description

You will design and execute evaluation tests to verify AI model safety, correctness, and adherence to guardrails.

Responsibilities

• Design and execute evaluation tests to detect hallucinations, assess factual correctness, evaluate tone, and verify safety guardrails for AI models.
• Build and maintain automated evaluation systems to score LLM responses.
• Conduct proactive red teaming to identify vulnerabilities like prompt injection and jailbreaks in AI systems.
• Validate the quality and diversity of training and validation datasets.
• Implement observability systems to detect model and feature drift in production.

Required Skills

• 3+ years of experience in QA, data validation, or ML testing.
• Hands-on exposure to at least one end-to-end AI pipeline from data to model to evaluation to deployment.
• Proficiency in Machine Learning, Python, Generative AI, LLM, and RAG.
• Experience with Test Automation and Data Validation practices.
• Familiarity with Statistical Analysis, including distributions and variance.
• Prior involvement in testing or validating AI-powered features or ML models.
• Bachelor's degree in Computer Science & Engineering, Software Engineering, Information Technology, Data Science, or related discipline.

Key Skills

Education

Any Graduate