{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad6ee91a-2f0c-5617-961b-cf61895cb5e5","task_key":"all--test--ad6ee91a-2f0c-5617-961b-cf61895cb5e5","task_revision_id":"1","upstream_id":"","short_description":"Generally, erotic materials are protected by the First Amendment for a person's…","config":"all","split":"test","body":"{\"choices\":\"['Whether the work, taken as a whole and judged by contemporary community standards, would be seen by a sensitive or vulnerable person to appeal to the prurient interest.', 'Whether the work depicts, in a direct or indirect way, sexual conduct specifically defined by the applicable state obscenity law.', 'Whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.', 'Whether the work has been properly judged from a national standard to offend sensitive standards of community morals.']\",\"question\":\"Generally, erotic materials are protected by the First Amendment for a person's personal use and possession. However, certain erotic material can reach a point where it steps over the boundaries of First Amendment protection. The Supreme Court has held that if material is obscene it does not mandate First Amendment protection. Which of the following most accurately represents one of the prongs of the three-pronged test enunciated by the U.S. Supreme Court's for determining whether a film, book, video or pictorial is obscene and therefore not subject to protection?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}