Role Summary
Audits first-pass text and multimedia ratings in an AI data workflow. The role resolves label disagreements and strengthens consistency between evaluators.
This published opportunity is available through Toloka AI. Review the listing carefully for the specific work arrangement and requirements.
Responsibilities
Audit relevance, user-intent and taxonomy labels. Re-score items and document discrepancies. Review linguistic accuracy and media comprehension. Track error categories, trends and difficult cases. Provide evaluator feedback and lead calibration. Report agreement and accuracy metrics.
Qualifications
U.S.-based; native or near-native English; strong grammar and proofreading; video/image comprehension and audio listening ability; understanding of search intent; consistent judgment and clear written rationales.
Preferred Qualifications
Data-labeling or query-annotation QA; sampling and calibration; root-cause analysis; spreadsheets and QA tools; short-video culture; linguistics, media or related studies.
Benefits
Not disclosed
What happens next
Submit your interest online to receive a reference number. The opportunity owner or review team may contact selected applicants about next steps.
For accommodation requests, contact [email protected].