{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9fb13af-1597-525a-aa04-5fa7953a2307","task_key":"all--test--f9fb13af-1597-525a-aa04-5fa7953a2307","task_revision_id":"1","upstream_id":"","short_description":"This jurisdiction has the following criminal statute in effect:\"A person is not…","config":"all","split":"test","body":"{\"choices\":\"['The defendant was suffering from temporary insanity.', 'The defendant lacked the requisite mental state required for the commission of the crime.', \\\"The defendant's failure to act was not voluntary.\\\", \\\"The defendant's criminal liability was superseded by that of the truck driver.\\\"]\",\"question\":\"This jurisdiction has the following criminal statute in effect:\\\"A person is not responsible for criminal conduct if at the time of such conduct, as a result of mental disease or defect, he lacks substantial capacity to appreciate the wrongfulness of his conduct, or to conform his conduct to the requirements of law. \\\"One afternoon, a defendant was babysitting his five-year-old nephew. As they were playing catch outside, the defendant threw the ball over his nephew's head, and it rolled into the street. Instinctively, the nephew ran after the ball but tripped over the gutter and fell in the street. When he tripped, the nephew severely sprained his ankle and couldn't stand up. Moments later, a large garbage truck was backing up and ran over the nephew, killing him. Although the defendant saw his nephew's predicament, he made no effort to rescue him. Subsequently, the defendant was charged with involuntary manslaughter. At trial, the defendant testified that he was so shocked when he saw his nephew fall near the garbage truck that he froze and was unable to move until it was too late. Following the defendant's testimony, an expert witness testified for the defense that reactions of the sort described by the defendant are not unusual when a truly shocking event occurs. If the jury believes the testimony of the defendant and his expert witness, the defendant's best defense is which of the following?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}