Description

You will review AI-scored sales call transcripts to validate quality rubrics, calibrate scoring consistency, and drive prompt refinement through systematic error analysis.

This role is remote.

Responsibilities

• Evaluate representative samples of AI-scored Mid-Market sales transcripts against defined quality skill rubrics.
• Investigate calls flagged by the governance system for potential scoring issues and log outcomes in the review tool.
• Complete calibration exercises using anchor examples to maintain inter-rater reliability and consistency.
• Categorise scoring discrepancies across governance pillars (Score Accuracy, Bias & Fairness, Drift) to identify systemic patterns.
• Communicate findings to support the prompt engineering cycle and expand the ground truth dataset for model benchmarking.

Required Skills

• 5+ years of experience in QA, auditing, or quality review, ideally in sales or contact centre environments.
• Strong analytical skills with a demonstrated attention to detail and ability to apply rubrics consistently.
• Experience evaluating sales conversations (calls, transcripts, or recordings) against structured quality frameworks.
• Comfort critically assessing AI-generated outputs and automated scores rather than defaulting to them.
• Strong written communication skills for documenting findings and flagging patterns clearly.
• Bachelor's degree.
• Self-directed and reliable in a remote working environment with minimal supervision.

Preferred Skills

• Experience with LLM evaluation, AI output review, or human-in-the-loop quality processes.
• Background in sales coaching, enablement, or sales methodology (e.g., MEDDIC, Challenger, SPIN).
• Proficiency in multiple languages relevant to EMEA Mid-Market sales.

Key Skills

Education

Bachelor's degree