{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1ef184c-d256-5009-9f32-fd749a86151f","task_key":"all--test--b1ef184c-d256-5009-9f32-fd749a86151f","task_revision_id":"1","upstream_id":"","short_description":"A state law prohibits any barbershop licensed by the state from displaying…","config":"all","split":"test","body":"{\"choices\":\"['No, because it treats barbershops differently from other commercial establishments.', 'No, because it imposes a restriction on the content or subject matter of speech in the absence of any evidence that such a restriction is necessary to serve a compelling state interest.', 'Yes, because it leaves political candidates free to communicate their campaign messages to voters by other means.', 'Yes, because the operation of a licensed barbershop is a privilege and therefore is subject to any reasonable restriction imposed by the state.']\",\"question\":\"A state law prohibits any barbershop licensed by the state from displaying posters in support of any current candidate for public office or displaying or distributing any campaign literature in support of such a candidate. No other kinds of posters or literature are subject to this prohibition, nor are any other types of commercial establishments in the state subject to similar prohibitions. Is this law constitutional?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}