{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64beadd7-dd57-50ce-90dc-29521db80f3f","task_key":"all--test--64beadd7-dd57-50ce-90dc-29521db80f3f","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued a defendant for leg injuries stemming from a collision between…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because the defendant admitted his negligence.', 'admissible, because it is a declaration against interest.', 'inadmissible, because the prejudice in admitting the testimony outweighs its probative value.', 'inadmissible, because an offer to pay medical bills is not admissible to prove negligence.']\",\"question\":\"A plaintiff sued a defendant for leg injuries stemming from a collision between their cars at an intersection. Each party was driving his own car, and the plaintiff alleged that the defendant had sped through a red light and collided with the plaintiff's car. The defendant denied this allegation. During the trial, the plaintiff testified that he and the defendant had conferred after the accident and that the defendant had said, \\\"Don't worry about anything; I'll see to it that your doctor's bills are paid. \\\" The defendant's attorney immediately objected. The trial judge should rule this testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}