{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20f33002-861d-5a1b-bbc6-2e161c223f4b","task_key":"default--test--1053","task_revision_id":"1","upstream_id":"1053","short_description":"What was the particular advantage of UNSC Resolution 678 over a collective…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Resolution 678 allowed for the use of force without any geographic restrictions.', 'Resolution 678 is broader because it authorised the use of force with respect to any matter threatening the peace and security of the Gulf region.', 'Resolution 678 overruled Article 51 UN Charter completely.', 'Resolution 678 provided an unlimited time frame for military action.', 'Resolution 678 is narrower than the inherent right of self-defence under Article 51 UN Charter', 'Resolution 678 mandated a ceasefire after a specific period.', 'Resolution 678 allowed unrestricted air strikes against Iraq.', 'Resolution 678 bypassed the need for UN Charter approval.', 'Resolution 678 allowed the allies to attack Iraq with nuclear and chemical weapons', 'Resolution 678 was faster to adopt and negotiate.']\",\"question\":\"What was the particular advantage of UNSC Resolution 678 over a collective defensive action under Article 51 UN Charter during the Gulf War? \",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}