{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1cfac33d-1b5c-5c24-800c-29df2b577b2c","task_key":"default--test--1600","task_revision_id":"1","upstream_id":"1600","short_description":"A federal statute provides states with funds for child welfare programs, subject…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because Congress can place any condition on the receipt of federal funds.', 'No, because the Eleventh Amendment does not bar actions brought by the United States.', 'Yes, because the Eleventh Amendment bars actions against a state in federal court.', 'Yes, because the protection of child welfare is reserved to the states.']\",\"question\":\"A federal statute provides states with funds for child welfare programs, subject to the condition that such programs be administered in accordance with federal standards. The United States sued a state in federal court for injunctive relief, arguing that the state's child welfare programs, which were funded in part by federal funds disbursed under this statute, failed to comply with federal standards. The state has moved to dismiss the action. Is the court likely to dismiss the action?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}