Description

Design and implement scalable automated testing frameworks and evaluation systems for content quality across Microsoft 365 and Copilot experiences.

This role is remote.

Responsibilities

• Design scalable testing and evaluation frameworks for content, templates, and AI-generated experiences.
• Develop automated testing workflows and validation tools to reduce manual effort while maintaining quality standards.
• Establish human-in-the-loop evaluation frameworks and quality rubrics for assessing AI outputs.
• Analyze testing metrics and operational data to identify improvement opportunities and optimize workflows.
• Partner with cross-functional teams to define testing strategies and drive adoption of quality standards.

Required Skills

• 2-4 years of experience in test automation, quality engineering, or systems development.
• Strong proficiency in Python and automation tooling.
• Experience with AI tooling, specifically Copilot and VSCode.
• Proven ability to translate data findings into strategic recommendations and actionable milestones.
• Experience building testing frameworks, quality systems, or workflow automation solutions.
• Strong technical problem-solving and systems-thinking skills.

Preferred Skills

• Experience evaluating generative AI systems or LLM-powered products.
• Familiarity with content quality measurement and content operations.
• Experience building internal tools, dashboards, or quality monitoring systems.

Key Skills

Education

Any Graduate