{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"009d00e2-1224-5005-9d3f-7e280c11be5f","task_key":"all--test--009d00e2-1224-5005-9d3f-7e280c11be5f","task_revision_id":"1","upstream_id":"","short_description":"A state law provides for an award of damages against anyone who publishes the…","config":"all","split":"test","body":"{\"choices\":\"['No, because the First Amendment prohibits the imposition of liability for the publication of truthful information.', 'No, because the judgment of the state supreme court rests upon an adequate and independent state-law ground.', 'Yes, because the supremacy clause does not permit a state to create rights greater than those conferred by the federal Constitution.', \\\"Yes, because the U.S. Supreme Court's appellate jurisdiction extends to cases arising under federal law.\\\"]\",\"question\":\"A state law provides for an award of damages against anyone who publishes the name of a rape victim. Pursuant to that law, a woman sued a local newspaper in state court after the newspaper identified her as a rape victim. The state trial and appellate courts rejected the claim, holding that the state law was invalid under both the state constitution and the First Amendment of the U.S. Constitution. The state supreme court affirmed, holding specifically: \\\"We think that this well-intentioned law very likely violates the First Amendment of the federal Constitution. We need not, however, decide that issue, because the law assuredly violates our state constitution, which provides even greater protection to the right of the press to report the news.\\\" The woman petitioned for review in the U.S. Supreme Court. Is the U.S. Supreme Court likely to review the state supreme court judgment?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}