{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"55d3ee9f-f233-5c74-be82-0f48a0146bf7","task_key":"all--test--55d3ee9f-f233-5c74-be82-0f48a0146bf7","task_revision_id":"1","upstream_id":"","short_description":"A surgeon performed an appendectomy on a patient in an operating room at the…","config":"all","split":"test","body":"{\"choices\":\"['admissible, provided that the judge determines that the surgeon destroyed the x-rays as a cover-up.', \\\"admissible, but leave the weight of the nurse's testimony to be determined by the jury.\\\", 'inadmissible, because the probative value is substantially outweighed by the danger of unfair prejudice.', 'inadmissible, because it is extrinsic evidence of a collateral matter.']\",\"question\":\"A surgeon performed an appendectomy on a patient in an operating room at the local hospital. The surgeon was assisted by a nurse who was assigned to the operation by the hospital. During the patient's early convalescence, he complained of pain not explicable as an ordinary post-operative symptom. On investigation, it turned out that the surgeon, who had bandaged him following the operation, had done so in such a manner as to constrict certain blood vessels. The faulty bandaging had caused acute pain and retarded the patient's recovery. After the patient's eventual recovery, he sued the surgeon for malpractice, claiming $25,000 in damages. In his case-in-chief, the patient called the nurse to testifS' that shortly after the surgery, she saw the surgeon destroy the postoperative x-rays of the patient's abdomen. Upon objection by the surgeon's attorney, the trial judge should rule the nurse's testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}