{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ea10b8f-ac3d-5998-88f5-2a34eeb7296f","task_key":"all--test--7ea10b8f-ac3d-5998-88f5-2a34eeb7296f","task_revision_id":"1","upstream_id":"","short_description":"What challenges to liberalism were shown by the Western response to the Arab…","config":"all","split":"test","body":"{\"choices\":\"['The problematic contention that the liberalist perspective is based on ideas that are Western - such as democracy, free enterprise, and civil rights.', 'Liberalism did not provide a clear way to deal with the Arab Spring, and its inconsistencies and contradictions were exposed.', 'The liberalist perspective does not explain clearly how and when liberalism is supposed to spread, in this case to the Arab countries in question.', 'All of the above']\",\"question\":\"What challenges to liberalism were shown by the Western response to the Arab Spring?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}