Role Summary
Assess language-model reasoning and reliability across realistic tasks. Produce reproducible evidence of errors and improve evaluation criteria for factuality, grounding, and task completion.
This published opportunity is available through Toloka AI. Review the listing carefully for the specific work arrangement and requirements.
Responsibilities
Check factual accuracy and logical consistency. Build rubrics, test plans, and regression suites. Record reproducible failures and likely causes. Evaluate tool use and retrieved-source faithfulness. Help automate testing and quality dashboards.
Qualifications
AI quality-assurance expertise; clear explanations of evaluation reasoning; secure computer and high-speed internet.
Preferred Qualifications
Relevant degree; ML quality assurance, red teaming, test automation, Python/SQL, and LLM evaluation-tool experience.
Benefits
No company-sponsored health insurance or paid time off.
What happens next
Submit your interest online to receive a reference number. The opportunity owner or review team may contact selected applicants about next steps.
For accommodation requests, contact [email protected].