{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbfcf4b7-a9f6-589b-8c27-18eeb717f5cb","task_key":"all--test--bbfcf4b7-a9f6-589b-8c27-18eeb717f5cb","task_revision_id":"1","upstream_id":"","short_description":"A defendant is on trial for attempted murder. The alleged victim is called by…","config":"all","split":"test","body":"{\"choices\":\"['admissible as an excited utterance.', 'admissible as a dying declaration.', 'inadmissible as hearsay not within any recognized exception.', 'inadmissible, because the victim was not questioned about the statement before being excused.']\",\"question\":\"A defendant is on trial for attempted murder. The alleged victim is called by the prosecution to testify. During her testimony, the victim recounted the incident and described how the defendant savagely beat her with a baseball bat. She was not asked by the prosecution whether she made any statements during the attack. After the victim was excused and left the witness stand, the prosecution called another witness to testify. The witness proposes to testify that when the beating stopped, the victim screamed, \\\"I'm dying. Don't let the defendant get away with this. \\\"Upon objection by the defendant's attorney, the witness's proffered testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}