{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ad996ce-0a87-5d6d-91be-5a387bb34b90","task_key":"all--test--3ad996ce-0a87-5d6d-91be-5a387bb34b90","task_revision_id":"1","upstream_id":"","short_description":"West posits that one reason that these harms go unnoticed in the U.S. is that…","config":"all","split":"test","body":"{\"choices\":\"['are less sensitive to threats from coercion.', 'are in general more sexist than other places.', 'tend to equate what is legal with what is good or harmless.', 'West actually thinks that people in the U.S. disproportionately recognize these harms.']\",\"question\":\" West posits that one reason that these harms go unnoticed in the U.S. is that people in the U.S.\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}