{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5f1b0bc-526f-5d9a-a512-6584f77f08ab","task_key":"default--test--5970","task_revision_id":"1","upstream_id":"5970","short_description":"Which of the following best defines federalism?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['A system where the national government controls all subnational units', \\\"The division of a country's power into national and subnational units\\\", 'A government system where power is primarily held by subnational units', 'The division of government power among different political parties', 'The division of national power into different branches of government', 'The accumulation of all government power within a single unit', 'A government in which the political leader is also the religious leader', 'The sharing of power between the national government and international organizations', 'The concentration of government power in the hands of a small group of people', 'The transfer of power from national to subnational units']\",\"question\":\"Which of the following best defines federalism?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}