{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfa865a2-df8b-5a0d-ad65-bed816d22bca","task_key":"all--test--bfa865a2-df8b-5a0d-ad65-bed816d22bca","task_revision_id":"1","upstream_id":"","short_description":"A defendant shot and killed a victim at close range with a revolver. After the…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The victim, who was the defendant's brother, suffered from an incurable case of cancer and asked the defendant to kill him to put an end to his pain and suffering.\\\", 'The killing was not done with premeditation or deliberation.', 'The defendant intended to kill himself, but the bullet grazed his head and struck and killed the victim.', 'The defendant intended to kill another man, but the victim unknowingly stepped in front of the other man and was struck and killed by the bullet.']\",\"question\":\"A defendant shot and killed a victim at close range with a revolver. After the shooting death, the defendant was arrested and charged with first-degree murder. In this jurisdiction, first-degree murder is defined as \\\"knowingly or willfully causing the death of another human being. \\\"Which of the following situations, if believed by the jury, would most likely result in the defendant's acquittal of the first-degree murder charge?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}