{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6ec5ce4-38fb-58e5-9299-063491da3d3e","task_key":"all--test--f6ec5ce4-38fb-58e5-9299-063491da3d3e","task_revision_id":"1","upstream_id":"","short_description":"Congress passed a federal statute that prohibited all \"indecent\" and \"patently…","config":"all","split":"test","body":"{\"choices\":\"['The law violates the First Amendment because it does not precisely regulate the content of speech and can be broadly applied to eliminate free speech for adults.', 'The law is a precise and narrow proscription that applies only to pornographic materials and its wording will be approved because it is necessary to protect children.', 'The law is constitutional because it is a reasonable exercise of the police power of the federal government.', 'The law is unconstitutional because it invades the absolute right to privacy that adults expect to be protected when using the Internet.']\",\"question\":\"Congress passed a federal statute that prohibited all \\\"indecent\\\" and \\\"patently offensive\\\" broadcasts on the Internet. The purpose of the statute was to protect children from accessing pornographic materials. The law imposed a blanket prohibition on any communications that could be accessed by children and which might contain the feared material. A group of Internet publishers filed an action to have the law declared unconstitutional and unenforceable. They proved that many instances of artistic expression and non-prurient broadcasts could fall within the wide parameters of the law. The government argued that its interest in protecting children outweighed the other considerations, and that millions of parents who are now afraid to access the Internet would use the service because the new law made it safe for children. What will the Supreme Court decide?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}