{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0faa3924-2e00-5ba8-8131-aef9bed5ed30","task_key":"all--test--0faa3924-2e00-5ba8-8131-aef9bed5ed30","task_revision_id":"1","upstream_id":"","short_description":"A defendant, a nurse at a nursing home, is charged with murdering a resident at…","config":"all","split":"test","body":"{\"choices\":\"['No, because the treatise excerpts were not offered during the examination of a qualified expert.', 'No, because the treatise itself must be introduced as an exhibit.', 'Yes, although hearsay, under the learned treatise exception to the hearsay rule.', \\\"Yes, because the lethal nature of the substance is relevant to the defendant's state of mind and intent.\\\"]\",\"question\":\"A defendant, a nurse at a nursing home, is charged with murdering a resident at the home by adding an allegedly lethal substance to the resident's food. At trial, to prove that the substance added to the resident's food could result in death, the prosecutor, without first calling any witnesses, offers to read into evidence several pages from a standard medical treatise that support the prosecution's claim that the substance the defendant added to the food is lethal. Is the evidence offered admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}