{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e1d7c91-1091-513f-a582-eb5d79faf246","task_key":"all--test--8e1d7c91-1091-513f-a582-eb5d79faf246","task_revision_id":"1","upstream_id":"","short_description":"A defendant was playing cards with a group of friends at their weekly poker…","config":"all","split":"test","body":"{\"choices\":\"['The defendant is correct with respect to assault, and the prosecution is correct with respect to battery.', 'The defendant is correct with respect to battery, and the prosecution is correct with respect to assault.', 'The prosecution is correct with respect to both the battery and assault charges.', 'The defendant is correct with respect to both the battery and assault charges.']\",\"question\":\"A defendant was playing cards with a group of friends at their weekly poker game. Earlier in the evening, one of the players handed the defendant a gun and told him it was unloaded. As a joke, the defendant pointed the gun at one of his friends and pulled the trigger. Unknown to the defendant, the gun was loaded and it discharged. The bullet grazed the friend's shoulder, slightly wounding him. The defendant was charged with assault with intent to kill. In this jurisdiction, battery is a lesser included offense of assault. At trial, the defendant requested that the court instruct the jury that if his mistake was honest, whether reasonable or unreasonable, it would be a valid defense to assault and the lesser included offense of battery. Conversely, the prosecution wanted the court to instruct the jury that in order for the defendant's mistake to be a valid defense for either crime, it must be reasonable. Regarding the jury instructions, which of the following statements is most accurate?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}