{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90d4f266-2787-5489-920d-4379dde7c1a5","task_key":"all--test--90d4f266-2787-5489-920d-4379dde7c1a5","task_revision_id":"1","upstream_id":"","short_description":"Considering Wendt's three cultures of anarchy; which of the following defines…","config":"all","split":"test","body":"{\"choices\":\"['A culture that exists some of the time, but not all the time, in which no self-restraint exists and actors cannot rely on each other for help.', 'A culture characterized by rivalry, which has dominated since the beginning of the modern system of states and in which actors regard each other as rivals but exercise some restraint in violence.', 'A culture characterized by friendship, where force and violence is eschewed in favour of cooperation in matters of security.', 'A culture within which survival relies on military power, security dilemmas abound, and security is a zero-sum game.']\",\"question\":\"Considering Wendt's three cultures of anarchy; which of the following defines the Lockean culture?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}