{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ea6218b-85f2-5918-85b0-ac5aa6ddd5a9","task_key":"all--test--9ea6218b-85f2-5918-85b0-ac5aa6ddd5a9","task_revision_id":"1","upstream_id":"","short_description":"A state has a statute generally prohibiting the installation and use of radar…","config":"all","split":"test","body":"{\"choices\":\"['constitutional, because it protects a legitimate state interest.', 'constitutional, because the commerce clause does not invalidate a state regulation relating to interstate commerce unless Congress takes express action to do so.', 'unconstitutional, because the state statute fails to give credit to the law of a neighboring state.', 'unconstitutional, because it unduly burdens interstate commerce.']\",\"question\":\"A state has a statute generally prohibiting the installation and use of radar detection devices in any motor vehicle operating on the roadways within the state. This prohibition was enacted to prevent motorists from evading radar or speed checkpoints on county and state highways. A neighboring state has no such regulation in effect. By the same token, Congress has taken no action on the matter. A resident of the neighboring state has installed a radar detection device in his automobile. While driving to visit his mother he is arrested and charged with violating the aforementioned state statute. The resident files a complaint challenging the constitutionality of the state statute. As applied to the resident, the state prohibition against the use of radar detection devices is likely to be held\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}