{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23c82722-d65d-5a2f-bf4b-acb8d023096e","task_key":"all--test--23c82722-d65d-5a2f-bf4b-acb8d023096e","task_revision_id":"1","upstream_id":"","short_description":"While riding her bicycle along the street, a woman was struck by a vehicle that…","config":"all","split":"test","body":"{\"choices\":\"['admissible as a statement of recent perception.', 'admissible as a present sense impression.', 'inadmissible, because the driver was available to testify at trial.', 'inadmissible as hearsay not within any recognized exception.']\",\"question\":\"While riding her bicycle along the street, a woman was struck by a vehicle that she didn't see. Subsequently, the woman sued the driver of the vehicle to recover damages for her injuries. At trial, the woman calls a police officer to testify that a few minutes after the accident, a driver stopped him and said, \\\"Hey, officer, I just saw an accident involving a red truck that hit this girl who was riding a bicycle. The truck left the scene of the accident and I followed it to a warehouse. \\\" The police officer then testified that he immediately drove to the warehouse and saw the defendant sitting in a red truck that was parked in the lot. The driver is available to testify at trial. Upon objection by the defendant's attorney, the police officer's testimony regarding the driver's statement should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}