{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dc372eb9-8fb9-5752-a70b-486fe55914ea","task_key":"all--test--dc372eb9-8fb9-5752-a70b-486fe55914ea","task_revision_id":"1","upstream_id":"","short_description":"A state legislature has recently enacted an obscenity statute prohibiting \"the…","config":"all","split":"test","body":"{\"choices\":\"['The particular materials involved depicted normal, not deviant, sexual conduct.', 'The particular materials involved consisted of serious scientific studies of human sexual urges.', 'The police did not have a search warrant when they entered the bookstore to purchase the particular materials involved in this obscenity prosecution.', '85% of the citizens of the state believe that the sale of sexually explicit material does not contribute to antisocial sexual behavior.']\",\"question\":\"A state legislature has recently enacted an obscenity statute prohibiting \\\"the selling or offering for sale of any obscene printed or video materials. \\\" Following numerous complaints from a local citizens' group, the police entered an adult bookstore, examined materials on the shelves, and purchased magazines that depicted actual pictures of sexual intercourse. The owner was subsequently arrested and charged with violating the state obscenity law. At his trial, the owner's defense was that the sale of the materials complained of was constitutionally protected speech. Which of the following, if established, would be most helpful to the owner's defense?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}