{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be05849b-4442-5835-a2a3-6b12d37b4a3b","task_key":"default--test--5288","task_revision_id":"1","upstream_id":"5288","short_description":"The swift adoption of the Bill of Rights in the years following ratification of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"Framers' unanimous agreement on all constitutional issues\\\", \\\"Framers' lack of commitment to individual rights\\\", \\\"states' indifference towards the national government\\\", \\\"states' fears of an overpowerful national government\\\", \\\"Framers' unqualified commitment to individual rights\\\", \\\"Northern states' support for abolitionism\\\", \\\"Northern and Southern states' compromise over representation\\\", \\\"small states' determination to receive equal representation in the legislature\\\", \\\"Southern states' support for slavery\\\", \\\"large states' push for more representation in the legislature\\\"]\",\"question\":\"The swift adoption of the Bill of Rights in the years following ratification of the Constitution demonstrates the\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}