{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c2415cc-ca40-50d3-ac23-16eb7224c2fd","task_key":"all--test--6c2415cc-ca40-50d3-ac23-16eb7224c2fd","task_revision_id":"1","upstream_id":"","short_description":"A defendant was angry at his friend for marrying the defendant's former…","config":"all","split":"test","body":"{\"choices\":\"['He intended to kill the friend and not the daughter.', 'He intended only to wound the friend.', 'He was unaware of the elements of malice.', 'The killing was the result of negligence in missing the friend.']\",\"question\":\"A defendant was angry at his friend for marrying the defendant's former girlfriend. As the friend was painting his house one afternoon, the defendant fired a shot from his gun at him. Although the shot missed the friend, the bullet struck and killed the friend's daughter. The defendant is subsequently charged with the first-degree murder of the daughter. The relevant statutes in effect in this jurisdiction are as follows:Section 169: Murder in the first degree is the unlawful and intentional killing of a human being with malice aforethought. Section 170: Malice is expressed when there is manifested a deliberate intention to take away the life of another. Which of the following, if established, would provide the defendant with his best defense?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}