Role Summary
Test AI responses across varied subjects and conversation styles. Develop clearer prompts and evaluation standards while documenting strengths, failures, and quality concerns.
This published opportunity is available through Toloka AI. Review the listing carefully for the specific work arrangement and requirements.
Responsibilities
Test new model versions against quality rubrics. Document successful and unsuccessful responses. Refine system prompts for tone and behavior. Review human and automated evaluations. Investigate failures and assess rubric effectiveness.
Qualifications
Five or more years in writing, editing, journalism, linguistics, STEM, coding, policy, or another relevant field; bachelor's degree or equivalent experience; subject expertise; English; careful rubric application.
Preferred Qualifications
At least one year of AI experience; generative-AI evaluation, fact-checking, source verification, and failure investigation.
Benefits
Health, dental, vision, life insurance, health savings account, paid time off, and retirement plan.
What happens next
Submit your interest online to receive a reference number. The opportunity owner or review team may contact selected applicants about next steps.
For accommodation requests, contact [email protected].