{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74c93c44-c414-5aa2-bdd4-e185b3ca07a0","task_key":"all--test--74c93c44-c414-5aa2-bdd4-e185b3ca07a0","task_revision_id":"1","upstream_id":"","short_description":"A state initiated a criminal prosecution against the owner of a video store for…","config":"all","split":"test","body":"{\"choices\":\"['No, because the store owner is engaged in the commercial sale of the video, which is not protected by the First and Fourteenth Amendments.', 'No, because the video consists entirely of portrayals of nudity, appeals to the prurient interest of viewers, and lacks serious social value as a whole.', 'Yes, because mere portrayals of nudity are insufficient to justify a finding that the video is obscene as a matter of constitutional law.', 'Yes, because the portrayals of nudity occurred outside the United States, and therefore the state lacks a compelling interest in applying its anti-obscenity law to the sale of the video.']\",\"question\":\"A state initiated a criminal prosecution against the owner of a video store for selling a video that consisted entirely of pictures of nude sunbathers on a beach in a foreign country where nude public sunbathing is common. The state charged that selling the video violated its anti-obscenity law. The store owner defended on the ground that the prosecution violated his constitutional right to freedom of speech. Should the store owner prevail in this defense?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}