{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4f58b9a-746e-5b47-957b-df4bf7d8b68e","task_key":"all--test--a4f58b9a-746e-5b47-957b-df4bf7d8b68e","task_revision_id":"1","upstream_id":"","short_description":"A staff assistant for a state agency was convicted in federal court of taking…","config":"all","split":"test","body":"{\"choices\":\"['the staff assistant was afforded an opportunity to express his views about the new legislation at public hearings, prior to the enactment of the statute.', 'deprivation of pension benefits is not cruel and unusual punishment.', 'a letter sent through ordinary mail is sufficient notice to satisfj due process for discontinuation of pension benefits.', \\\"it is implicit that one of the conditions of the state's contract of employment with a state agency staff member is that he shall not engage in bribery.\\\"]\",\"question\":\"A staff assistant for a state agency was convicted in federal court of taking bribes from a foreign government for the purpose of influencing an upcoming vote on a waterworks bill. He was sentenced to probation. The staff assistant had served in the agency long enough to become fully qualified for his pension upon retirement under the terms of an agreement between the pensions board and the union. The staff assistant retired and immediately started receiving monthly pension checks. Subsequently, the governor signed into law an act, which provided in part:\\\"Section 8. Any member of a state agency's staff who is convicted of. . . bribery . . . shall not beentitled to receive any retirement or other benefit or payment of any kind from the state . . . Such conviction shall be considered a breach of the staff member's employment contract. \\\"The staff assistant received a letter from the state which stated that pursuant to this new act the state is immediately discontinuing pension benefit payments to you on account of your bribery conviction. The staff assistant contacted an attorney, who challenged the discontinuance of benefits on the grounds that the new law was unconstitutional. To counter one of the attorney's possible arguments regarding the unconstitutionality of Section 8 of the act, the state's best rebuttal would be that\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}