{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"758b287e-9645-51a4-970f-01cdd9cc77a9","task_key":"default--test--1601","task_revision_id":"1","upstream_id":"1601","short_description":"A federal statute provides states with funds for child welfare programs, subject…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the state has sovereign immunity under the Tenth Amendment.', 'Yes, because the protection of child welfare is reserved to the states.', 'Yes, because the state has complied with the federal standards to the best of its ability.', 'No, because the state has failed to meet the conditions required to receive federal funds.', 'No, because the federal court has the authority to enforce compliance with federal standards.', 'Yes, because the Eleventh Amendment bars actions against a state in federal court.', 'No, because the Eleventh Amendment does not bar actions brought by the United States.', 'No, because the federal government has the power to enforce federal standards for programs they fund.', 'Yes, because the court lacks jurisdiction over disputes between the federal government and a state.', 'No, because Congress can place any condition on the receipt of federal funds.']\",\"question\":\"A federal statute provides states with funds for child welfare programs, subject to the condition that such programs be administered in accordance with federal standards. The United States sued a state in federal court for injunctive relief, arguing that the state's child welfare programs, which were funded in part by federal funds disbursed under this statute, failed to comply with federal standards. The state has moved to dismiss the action. Is the court likely to dismiss the action?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}