{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"603bf581-70d9-5615-b811-f4a6ec1e7c36","task_key":"all--test--603bf581-70d9-5615-b811-f4a6ec1e7c36","task_revision_id":"1","upstream_id":"","short_description":"While walking across the street one afternoon, a pedestrian was hit by a car and…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because the license plate identification substantiates that the other information is accurate and trustworthy.', 'admissible, because it is a statement of past recollection refreshed.', 'inadmissible, because it is hearsay not within any recognized exception.', 'inadmissible, because statements procured through hypnosis cannot be authenticated as truthful.']\",\"question\":\"While walking across the street one afternoon, a pedestrian was hit by a car and thrown down in the middle of the intersection. The driver of the vehicle did not stop. Although he never lost consciousness, the pedestrian was in excruciating pain and has been taking strong painkillers since the accident that have affected his memory of the accident. At trial, the pedestrian calls a renowned hypnotist to testify. The hypnotist proposes to testify that after he placed the pedestrian under a hypnotic spell, the pedestrian gave a detailed account of the accident and told the hypnotist that the driver ran a red light and that the driver's license plate number was GIJO-7 14. The driver concedes that his license plate number isGIJO-7 14. The hypnotist's proposed testimony will most likely be held to be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}