{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e61b7159-1fe6-562b-80c8-ab3636fbf777","task_key":"default--test--1491","task_revision_id":"1","upstream_id":"1491","short_description":"Which of these statements best describes the UK Constitution?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The UK Constitution is a direct copy of the US Constitution', 'The UK Constitution is codified and contained in a single document', 'The UK Constitution can be changed by a simple majority vote in Parliament', 'The UK Constitution gives the judiciary the power to overturn acts of parliament', 'The UK Constitution is based on a Bill of Rights', \\\"The UK Constitution's power is derived solely from Parliament\\\", 'The UK Constitution is based entirely on European Union law', 'The UK Constitution is uncodified and can be found in a number of sources', \\\"The UK Constitution's only source of power is that of the sovereign\\\", 'The UK Constitution is based solely on common law']\",\"question\":\"Which of these statements best describes the UK Constitution?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}