{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a0b8f9b-5af6-58ae-86b1-38d20274da4c","task_key":"all--test--3a0b8f9b-5af6-58ae-86b1-38d20274da4c","task_revision_id":"1","upstream_id":"","short_description":"A defendant was arrested and charged with involuntary manslaughter for causing a…","config":"all","split":"test","body":"{\"choices\":\"['granted, because the statement is hearsay not within any recognized exception.', 'granted, because the questioning of the witness was leading.', \\\"denied, because the defendant's attorney did not object at the grand jury hearing that the witness's testimony was hearsay.\\\", \\\"denied, because the defendant's attorney did not object at the grand jury hearing that the questioning of the witness was leading.\\\"]\",\"question\":\"A defendant was arrested and charged with involuntary manslaughter for causing a vehicular homicide. A grand jury was convened, and various witnesses were subpoenaed. At the grand jury, a witness was asked the following question by the prosecuting attorney: \\\"Is it not true that 20 minutes after the accident you told the police that the defendant's car went through the red light?\\\" He answered, \\\"Yes. \\\"The defendant was also subpoenaed to appear before the grand jury. The defendant was represented by counsel at the grand jury hearing. During questioning he was shown the witness's statement from his testimony. No objection was made to the introduction of the witness's statement. At the defendant's trial, the prosecuting attorney seeks to introduce into evidence the witness's statement from the deposition taken at the grand jury hearing. The defendant's attorney objects to this proffer of evidence. The objection at trial will be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}