{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc7c17cc-c582-5d39-9a17-48115a2fa5fc","task_key":"all--test--cc7c17cc-c582-5d39-9a17-48115a2fa5fc","task_revision_id":"1","upstream_id":"","short_description":"One year ago, while driving through an intersection, the defendant's car…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admitted, because the plaintiff's inconsistent statement has bearing on her credibility.\\\", 'admitted, because the accident investigator gathered the information in the course of his official duties.', 'excluded, because the defendant failed to lay a foundation, thus not giving the plaintiff an opportunity to deny or explain her statement.', \\\"excluded, because the plaintiff's statement to the investigator is hearsay.\\\"]\",\"question\":\"One year ago, while driving through an intersection, the defendant's car collided with the plaintiff's car. The plaintiff sued, claiming that the defendant ran a red light. In her case-in-chief, the plaintiff testified that immediately after the accident, a bystander, who was a local resident, screamed at the defendant, \\\"You jerk. . . you drove through the red light. \\\" In his case-in-chief, the defendant now calls a police accident investigator to testify that the plaintiff told him that she \\\"didn't see the color of the light when she entered the intersection because she was dialing a number on her cell phone. \\\"The investigator's testimony should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}