Description

You will build and maintain data systems to support model evaluation and training cycles.

Responsibilities

• Prototype human rating and semi-automatic model evaluation protocols with the team.
• Conduct small-to-medium scale in-house human rating for complex protocols.
• Identify and summarize model gaps in collaboration with engineers.
• Determine required data to address gaps and mine data to prepare model training sets.
• Scale validated evaluation protocols by coordinating and auditing with PDO teams.

Required Skills

• 5+ years of relevant professional experience.
• Basic knowledge of Python, SQL, and Unix.
• Hands-on experience with large language models (LLMs).
• Solid experience with data pipelines and data engineering workflows.
• Understanding of core Artificial Intelligence and Machine Learning concepts.
• Experience prompting models and interpreting their outputs.
• Familiarity with image data processing.
• Attention to detail and aptitude for experimental investigation.

Key Skills

Education

Any Gradute