{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a8474c5-df69-5bdf-ab17-fa9066bec8b2","task_key":"all--test--4a8474c5-df69-5bdf-ab17-fa9066bec8b2","task_revision_id":"1","upstream_id":"","short_description":"A federal statute authorizes a federal agency to issue rules requiring that…","config":"all","split":"test","body":"{\"choices\":\"['No, because the federal government may not compel a state legislature to enact into state law a federally mandated regulatory program.', 'No, because the Tenth Amendment grants states immunity from all direct federal states immunity from all direct federal regulation.', 'Yes, because the rules serve an important purpose, and the requirements they impose on the states are only temporary and do not excessively interfere with the functioning of the state governments.', 'Yes, because the supremacy clause of Article VI requires states to enforce federal law.']\",\"question\":\"A federal statute authorizes a federal agency to issue rules requiring that state legislatures adopt laws of limited duration to reduce water pollution from gasoline-powered boat motors. The purpose of these rules is to assist the agency in attaining the clean water standards required by the statute. After the agency issued such rules, several states filed an action challenging the rules on the sole ground that they are unconstitutional. Should the court uphold the constitutionality of the agency's rules?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}