{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"73ee8572-0579-56c7-9d9a-6a9af8e07744","task_key":"all--test--73ee8572-0579-56c7-9d9a-6a9af8e07744","task_revision_id":"1","upstream_id":"","short_description":"A city passed a picketing ordinance. Section 202 of that ordinance…","config":"all","split":"test","body":"{\"choices\":\"['constitutional, as a permissible time, place, and manner regulation.', \\\"constitutional, as a valid exercise of a state's reserved powers under the Tenth Amendment.\\\", 'unconstitutional, as discriminatory on its face.', 'unconstitutional, as vague and overbroad.']\",\"question\":\"A city passed a picketing ordinance. Section 202 of that ordinance reads:\\\"Section 202. It shall be unlawful for any person to engage in picketing before or about the residence of an individual. Nothing herein shall be deemed to prohibit the holding of a meeting or assembly on any premises used for the discussion of subjects of general public interest. \\\"A group of teachers who work for a public school inside of the city were angry that the principal decreased their salaries. The teachers reacted by picketing on the lawn of the principal's privately owned house. The teachers were charged with violating Section 202 of the ordinance. The teachers have filed suit attacking the constitutionality of Section 202. The court will most likely declare Section 202\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}