Toloka AI / careers
← All positions

Model evaluation · Toloka AI

Content Specialist III | AI Evaluation & Prompting

United StatesRemoteContract

Role Summary

Test AI responses across varied subjects and conversation styles. Develop clearer prompts and evaluation standards while documenting strengths, failures, and quality concerns.

This published opportunity is available through Toloka AI. Review the listing carefully for the specific work arrangement and requirements.

Responsibilities

Test new model versions against quality rubrics. Document successful and unsuccessful responses. Refine system prompts for tone and behavior. Review human and automated evaluations. Investigate failures and assess rubric effectiveness.

Qualifications

Five or more years in writing, editing, journalism, linguistics, STEM, coding, policy, or another relevant field; bachelor's degree or equivalent experience; subject expertise; English; careful rubric application.

Preferred Qualifications

At least one year of AI experience; generative-AI evaluation, fact-checking, source verification, and failure investigation.

Benefits

Health, dental, vision, life insurance, health savings account, paid time off, and retirement plan.

What happens next

Submit your interest online to receive a reference number. The opportunity owner or review team may contact selected applicants about next steps.

For accommodation requests, contact [email protected].