{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1977f3c0-721f-555b-aaf8-00ede2fc2907","task_key":"default--test--974","task_revision_id":"1","upstream_id":"974","short_description":"Is the ICJ hostile to the idea that an armed attack may arise through a series…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The ICJ has often supported the accumulation theory', 'The ICJ considers each instance of armed force individually, not cumulatively', 'The ICJ has never addressed this issue', 'The concept of an armed attack refers to a single attack, not multiple smaller instances of armed force', 'The ICJ has shown mixed views on the accumulation theory', 'In the Nicaragua case the ICJ dismissed the accumulation theory', 'The ICJ was not hostile to the accumulation theory', 'The ICJ has consistently rejected the concept of an armed attack arising from cumulative instances', 'The ICJ has yet to make a definitive ruling on the accumulation theory', 'The ICJ ruled in favor of the accumulation theory in the Bosnia and Herzegovina case']\",\"question\":\"Is the ICJ hostile to the idea that an armed attack may arise through a series of lower intensity, yet cumulative, instances of armed force?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}