Description

You will build and maintain data systems to support model evaluation and training cycles.

This role is on-site.

Responsibilities

• Prototype human rating and semi-automatic model evaluation protocols.
• Conduct small-to-medium scale in-house human rating for complex protocols.
• Identify and summarize model gaps in collaboration with engineers.
• Mine data to prepare model training sets addressing identified gaps.
• Scale validated evaluation protocols by coordinating and auditing with PDO teams.

Required Skills

• 5+ years of relevant professional experience.
• Hands-on experience with large language models (LLMs).
• Solid experience with data pipelines and data engineering workflows.
• Proficiency in Python, SQL, and Unix.
• Understanding of core Artificial Intelligence and Machine Learning concepts.
• Experience prompting models and interpreting their outputs.
• Familiarity with image data processing.

Preferred Skills

• Attention to detail and aptitude for experimental investigation.

Key Skills

Education

Any Gradute