Description

You will architect and execute scalable test automation solutions for Generative AI and Agentic AI applications, ensuring quality and reliability in production environments.

This role is on-site.

Responsibilities

• Design and implement testing strategies for LLMs, including NLU/NLP validation, bias detection, and responsible AI practices.
• Architect enterprise-scale cloud-based testing solutions on AWS, leveraging Bedrock, Lambda, and CloudWatch.
• Optimize CI/CD pipelines using Jenkins or GitLab CI, integrating quality gates and governance frameworks for Gen AI.
• Evaluate RAG systems using vector databases and frameworks like RAGAS or DeepEval.
• Collaborate with cross-functional teams to establish testing excellence and drive quality initiatives across the SDLC.

Required Skills

• 6+ years of experience with Python programming and scalable test automation architecture.
• Deep hands-on experience with Generative AI, Agentic AI, LLMs, LangChain, and LangGraph.
• Proficiency in NLP/NLU testing methodologies and prompt engineering evaluation.
• Strong knowledge of CI/CD pipelines, SDLC best practices, and DevOps culture.
• Experience with version control (Git, Stash), Jira, and test management tools (Xray, TestRail).
• Familiarity with RAG testing, vector databases, and performance/load testing for Gen AI.
• Understanding of Agile/Scrum methodologies and cross-functional collaboration.

Preferred Skills

• Experience with containerization (Docker, Kubernetes) for test environments.
• Background in data engineering or data quality testing.

Key Skills

Education

Any Graduate