{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f000630-5cc1-50e3-852a-bd546dbdcaac","task_key":"all--test--8f000630-5cc1-50e3-852a-bd546dbdcaac","task_revision_id":"1","upstream_id":"","short_description":"A defendant was on the first day of her new secretarial job when her boss called…","config":"all","split":"test","body":"{\"choices\":\"['win, because the statute is unconstitutional.', 'win, because the statute violates due process.', 'lose, because she failed to overcome the presumption.', 'lose, because the presumption may be rebutted.']\",\"question\":\"A defendant was on the first day of her new secretarial job when her boss called her into his office. The boss directly suggested that if the defendant did not go out on a date with him, she would be fired in one week. Every day during the remainder of the week, the boss approached the defendant with his demand, and the defendant refused to cooperate. At the end of the week, when the boss called the defendant into his office and again tried to pressure her to go out on a date with him, the defendant knocked him unconscious with a giant stapler and choked him to death. The defendant is tried for murder. In accordance with the following statute, the state relies at trial on the presumption of malice:\\\"When the act of killing another is proved, malice aforethought shall be presumed, and the burden shall rest upon the party who committed the killing to show that malice did not exist. \\\"If the defendant is convicted of first-degree murder and challenges her conviction on the grounds of the above statute, on appeal she will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}