{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"310737a1-1110-5efb-a278-1b466c7af77f","task_key":"all--test--310737a1-1110-5efb-a278-1b466c7af77f","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff was seriously injured in a collision involving three trucks. A…","config":"all","split":"test","body":"{\"choices\":\"['admissible as a declaration of existing state of mind.', 'admissible as an excited utterance.', 'inadmissible, because it contains inadmissible opinion evidence.', 'inadmissible as hearsay not within any recognized exception.']\",\"question\":\"A plaintiff was seriously injured in a collision involving three trucks. A sideswipe occurred between the plaintiff's pickup truck and a trailer truck as they passed in opposite directions. The sideswipe caused the plaintiff's truck to careen down the road and into the path of a second trailer truck. The driver of the second trailer truck was immediately killed. Both of the trailer trucks were owned by the same transport company. At trial, the plaintiff called a witness to the collision to testify that he heard an unidentified female witness scream, \\\"Oh my God, that trailer truck sideswiped that little pickup truck. \\\"The trial judge should rule the witness's testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}