{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbc3a1d8-ac12-51c1-a0bc-95e0c5a75d59","task_key":"all--test--bbc3a1d8-ac12-51c1-a0bc-95e0c5a75d59","task_revision_id":"1","upstream_id":"","short_description":"A federal statute provides states with funds for child welfare programs, subject…","config":"all","split":"test","body":"{\"choices\":\"['No, because Congress can place any condition on the receipt of federal funds.', 'No, because the Eleventh Amendment does not bar actions brought by the United States.', 'Yes, because the Eleventh Amendment bars actions against a state in federal court.', 'Yes, because the protection of child welfare is reserved to the states.']\",\"question\":\"A federal statute provides states with funds for child welfare programs, subject to the condition that such programs be administered in accordance with federal standards. The United States sued a state in federal court for injunctive relief, arguing that the state's child welfare programs, which were funded in part by federal funds disbursed under this statute, failed to comply with federal standards. The state has moved to dismiss the action. Is the court likely to dismiss the action?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}