{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5de7936d-6858-57d8-866e-8979dd9794f9","task_key":"all--test--5de7936d-6858-57d8-866e-8979dd9794f9","task_revision_id":"1","upstream_id":"","short_description":"The primary advantage of the forced-choice distribution of rated attributes in…","config":"all","split":"test","body":"{\"choices\":\"['eliminates statistical concerns about ipsative measurement', 'enhances the reliability of ratings', 'is more objective, since it does not deal with the strengths and weaknesses of the employee', 'is easier and faster to use, since only two choices are available']\",\"question\":\"The primary advantage of the forced-choice distribution of rated attributes in performance appraisal is that it\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}