{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40e7ecd9-5e39-56cd-af64-bb35c374c70c","task_key":"all--test--40e7ecd9-5e39-56cd-af64-bb35c374c70c","task_revision_id":"1","upstream_id":"","short_description":"Following their law school graduation party, a graduate and his roommate were…","config":"all","split":"test","body":"{\"choices\":\"['admissible as an admission.', 'admissible as a declaration against interest.', 'admissible under the former testimony exception to the hearsay rule.', 'inadmissible, because the suit was dismissed as part of a compromise.']\",\"question\":\"Following their law school graduation party, a graduate and his roommate were driving home in an automobile that was owned and operated by the graduate. As they were approaching an intersection, their vehicle collided with a motorcycle whose rider was seriously injured in the accident. The rider sued the graduate for operating his motor vehicle in excess of the speed limit. In addition, the roommate brought suit against the rider for operating his motorcycle negligently and driving through a red light. The roommate subsequently reached an out- of-court settlement with the graduate and dismissed the lawsuit against him. At trial against the rider, the roommate testified that he observed the rider drive his motorcycle through a red light and then swerve into the path of the graduate's automobile. In rebuttal, the rider offered into evidence the roommate's complaint against the graduate alleging that the latter was driving \\\"at an excessively high rate of speed at the time the accident occurred. \\\"The complaint is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}