{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7f79b2f-46ee-56cd-aa47-8ad3c411dda6","task_key":"all--test--e7f79b2f-46ee-56cd-aa47-8ad3c411dda6","task_revision_id":"1","upstream_id":"","short_description":"What was the particular advantage of UNSC Resolution 678 over a collective…","config":"all","split":"test","body":"{\"choices\":\"['Resolution 678 allowed the allies to attack Iraq with nuclear and chemical weapons', 'Resolution 678 is narrower than the inherent right of self-defence under Article 51 UN Charter', 'Resolution 678 was faster to adopt and negotiate.', 'Resolution 678 is broader because it authorised the use of force with respect to any matter threatening the peace and security of the Gulf region.']\",\"question\":\"What was the particular advantage of UNSC Resolution 678 over a collective defensive action under Article 51 UN Charter during the Gulf War?\",\"subject\":\"international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}