{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d77f93d-a47d-5394-b89e-d3b42d5c09fa","task_key":"all--test--3d77f93d-a47d-5394-b89e-d3b42d5c09fa","task_revision_id":"1","upstream_id":"","short_description":"A patient was admitted to a hospital. He had driven his car despite being…","config":"all","split":"test","body":"{\"choices\":\"[\\\"a cause-in-fact, but not a legal cause of the patient's death.\\\", \\\"a legal cause, but not a cause-in-fact of the patient's death.\\\", \\\"a cause-in-fact and a legal cause of the patient's death.\\\", \\\"neither a legal cause nor a cause-in-fact of the patient's death.\\\"]\",\"question\":\"A patient was admitted to a hospital. He had driven his car despite being intoxicated. As a result, he crashed the car into a tree and suffered a broken nose. After a doctor reset the broken nose, the patient was transferred to a room in the west wing of the hospital. In extreme pain, the patient asked the nurse for a painkiller. Without seeking the doctor's approval, the nurse administered an injection of morphine, which the nurse should have known to be an excessive dosage. The patient died an hour after the injection; the cause of death was a morphine overdose. The nurse's injection of morphine to this patient would most likely constitute\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}