{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dbac4a5f-d080-5544-80e9-f63ab9576cf6","task_key":"all--test--dbac4a5f-d080-5544-80e9-f63ab9576cf6","task_revision_id":"1","upstream_id":"","short_description":"The defendant has been charged with murder in the killing of his wife. The…","config":"all","split":"test","body":"{\"choices\":\"['The defendant acted several days after the provoking events.', 'The defendant acted in an overpowering heat of passion.', 'The defendant was provoked by accidentally seeing his wife having sexual relations with his best friend.', 'The defendant acted without deliberation and premeditation.']\",\"question\":\"The defendant has been charged with murder in the killing of his wife. The defense attorney is preparing a defense to show facts sufficient to support a verdict of voluntary manslaughter. Counsel has to prove all of the necessary elements during trial in order to support a voluntary manslaughter charge to the jury. Which of the following would not be proper proof to establish a voluntary manslaughter charge to the jury?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}