{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3daab57b-73fb-5efc-ab29-a828f9fe88fa","task_key":"all--test--3daab57b-73fb-5efc-ab29-a828f9fe88fa","task_revision_id":"1","upstream_id":"","short_description":"While a ski lift was ascending the mountain, the overhead cable broke, dropping…","config":"all","split":"test","body":"{\"choices\":\"['admissible as former testimony.', \\\"admissible, provided that the witness's statements are more probative on the point than any other evidence the skier can procure through reasonable efforts and that the skier has given the ski resort reasonable notice that she would offer the deposition.\\\", \\\"inadmissible, because it violates the ski resort's right to confront the witnesses against itself.\\\", 'inadmissible, because it is hearsay not within any recognized exception.']\",\"question\":\"While a ski lift was ascending the mountain, the overhead cable broke, dropping a skier 15 feet to the ground. The skier suffered a broken pelvis on account of the fall. Thereafter, the skier sued the ski resort alleging negligent maintenance of the ski lift machinery. The ski resort's skier's attorney took a discovery deposition of a witness who was seated behind the skier at the time of the accident. The witness died shortly after the deposition. At trial, the skier offers the witness's deposition testimony into evidence. Upon objection by the ski resort's attorney, the deposition is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}