{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b34afd46-729f-519d-b256-243303ade49a","task_key":"all--test--b34afd46-729f-519d-b256-243303ade49a","task_revision_id":"1","upstream_id":"","short_description":"A nightclub owner applied for a required zoning permit to open a nude-dancing…","config":"all","split":"test","body":"{\"choices\":\"['No, because nude dancing is symbolic speech and is therefore protected by the First and Fourteenth Amendments.', \\\"No, because the organization does not seek a reasonable time, place, and manner regulation of nude dancing, but instead seeks a total ban on the owner's opening any type of nude-dancing facility at any time or in any place in the city.\\\", 'Yes, because the action is not ripe.', 'Yes, because the First and Fourteenth Amendments do not protect obscenity, and nude dancing is obscene.']\",\"question\":\"A nightclub owner applied for a required zoning permit to open a nude-dancing nightclub in the theater district of a city. An organization of influential city residents began an intensive lobbying effort to persuade the city council to deny the owner a permit to operate any type of nude- dancing facility at any time or in any place in the city. The owner has sued the city in an appropriate federal court, seeking an injunction that would prohibit the city council from considering the organization's views, on the ground that if the organization is successful in its lobbying efforts, the owner's First and Fourteenth Amendment rights would be violated. The city has moved to dismiss the action. Should the court dismiss the owner's action?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}