{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcc20328-ad93-5212-9f25-d4715e85f07c","task_key":"all--test--bcc20328-ad93-5212-9f25-d4715e85f07c","task_revision_id":"1","upstream_id":"","short_description":"An owner had a record store in the downtown business area of a city. A famous…","config":"all","split":"test","body":"{\"choices\":\"['the ordinance violates equal protection because some sounds are permitted, while others are not.', \\\"the ordinance violates the owner's rights of freedom of speech, because there is not valid interest to support the ordinance.\\\", \\\"the ordinance violates the owner's rights of freedom of speech, because a municipality may not regulate the use of sound amplification equipment.\\\", \\\"the ordinance violates the owner's rights under the First and Fourteenth Amendments, because it is vague in defining unpermitted sounds.\\\"]\",\"question\":\"An owner had a record store in the downtown business area of a city. A famous rock group was scheduled to perform at the local civic center and the owner featured the band's records in a special sale for the two weeks prior to the concert. In order to promote his sale, the owner installed loudspeakers on the outside of his store window so that he could play the band's records for people walking by to hear. It was the owner's hope that when they heard the records, the passersby would turn into customers and buy the band's records. Subsequently, the owner was cited for violating a city ordinance which provides that:\\\"An owner of property located within the city limits shall not permit to be used on his property any device which causes sounds, other than clock chimes, to be heard upon the street or sidewalk. Violation of this ordinance shall subject the property owner to a fine of $50. 00 for each occurrence. \\\"If the owner is successful in challenging this ordinance in court, the court would most likely reason that\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}