{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"365260d1-c91d-5d3a-ba8d-24363cbb94f7","task_key":"default--test--5362","task_revision_id":"1","upstream_id":"5362","short_description":"Which of the following is an example of devolution?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Allowing states to decide how to meet federally mandated greenhouse gas emissions targets', 'The federal government setting a minimum wage that all states must meet', 'A state being prevented from implementing a fuel efficiency standard that is more stringent than that set by the federal government', 'A \\\"race to the bottom\\\" in which states compete with one another to have the fewest environmental regulations', 'A state being forced to comply with federal immigration policies', 'The federal government establishing nationwide educational standards', 'A state choosing to implement stricter gun control laws than those mandated by the federal government', 'A state that decriminalizes all drug use, contrary to federal law', 'A state that legalizes marijuana possession in conflict with federal law', \\\"The federal government overriding a state's decision to legalize same-sex marriage\\\"]\",\"question\":\"Which of the following is an example of devolution?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}