{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29ffe34b-20fa-51cb-be69-c563abda3071","task_key":"all--test--29ffe34b-20fa-51cb-be69-c563abda3071","task_revision_id":"1","upstream_id":"","short_description":"A woman was married to a very wealthy man who had a generous life insurance…","config":"all","split":"test","body":"{\"choices\":\"['first-degree murder.', 'second-degree murder.', 'manslaughter.', 'no crime.']\",\"question\":\"A woman was married to a very wealthy man who had a generous life insurance policy. Deciding that she had had enough of married life, and desiring to spend more time with her boyfriend, she began to plan an early and unexpected demise for her husband. The woman devised a scheme whereby she would contaminate her husband's guacamole with a poisonous substance called strychnine. To facilitate her plan, she enlisted the help of her boyfriend. At first, the boyfriend adamantly refused to go along with her scheme, but when the woman threatened to break off their relationship, he reluctantly agreed. The boyfriend, a pre-med student, got the strychnine from the pharmacy by telling them that he was using it for an experiment with rats. The boyfriend then gave the strychnine to the woman, who sprinkled it in her husband's guacamole. The husband died from the poisoning. A statute in the jurisdiction provides: \\\"Any intentional and premeditated killing or one occurring during the perpetration of a felony, is murder in the first degree. Murder in the second degree is murder with intent to cause serious bodily harm. Manslaughter is an unlawful killing due to recklessness. \\\" Assault with intent to kill or to cause serious physical injury is a felony in the jurisdiction. There are no separate felonies relating to adulterating food or poisoning. The most serious crime for which the boyfriend can properly be convicted is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}