{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67c0d9ac-b912-50dc-a4b6-749f5fa5ce5f","task_key":"all--test--67c0d9ac-b912-50dc-a4b6-749f5fa5ce5f","task_revision_id":"1","upstream_id":"","short_description":"A city school board presented a bond issue to the public for $5 million to…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because he went too far in criticizing his own employer.', \\\"No, the teacher's remarks were void for vagueness and couldn't be enforced against him.\\\", 'Yes, teachers are civil servants who must refrain from political issues.', 'No, because teachers, as informed citizens, have a constitutional right to speak out on current issues of importance.']\",\"question\":\"A city school board presented a bond issue to the public for $5 million to modernize some city schools. During the public debate on the proposal, a middle school teacher had published two \\\"letters to the editor\\\" criticizing the \\\"duplicitous\\\" practices of certain school board members in using \\\"backroom politics\\\" to try and get the vote approved. He doubted the need for new expenditures and criticized the superintendent of schools for trying to influence teachers on the issue. After the bond issue passed, the school district brought internal charges against the teacher, and after a hearing he was fired. He brought a state court action claiming an unconstitutional interference with his First Amendment free speech rights. The claim was rejected, and the highest state appellate court upheld the termination. After hearing the case on appeal, will the United States Supreme Court uphold the teacher's firing?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}