{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c6228eb-acce-57b9-b37f-7c1028500287","task_key":"all--test--6c6228eb-acce-57b9-b37f-7c1028500287","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff was hit and injured by a falling branch on a sidewalk in front of a…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because the witness is unavailable to testify.', 'admissible, as relevant for the limited purpose of proving that the convenience store owner retained control over the sidewalk where the accident occurred.', 'inadmissible, because it is hearsay not within any recognized exception.', \\\"inadmissible, because of the Dead Man's Statute.\\\"]\",\"question\":\"A plaintiff was hit and injured by a falling branch on a sidewalk in front of a convenience store. The plaintiff calls a woman to testify. The woman proposes to testify that a witness who is now deceased told the woman that when he was walking past the convenience store one week before the plaintiff was hit by the branch, the witness saw an employee of the convenience store shoveling snow from the sidewalk in front of the store. The woman's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}