{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25b6ef7f-6e98-5e09-829c-93a681a37aca","task_key":"default--test--1976","task_revision_id":"1","upstream_id":"1976","short_description":"A surgeon performed an appendectomy on a patient in an operating room at the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"admissible, but only if the surgeon is given an opportunity to refute the nurse's testimony.\\\", 'inadmissible, because the probative value is substantially outweighed by the danger of unfair prejudice.', 'inadmissible, because it is extrinsic evidence of a collateral matter.', 'admissible, if the patient can demonstrate that the destroyed x-rays would have supported his claim.', \\\"admissible, but leave the weight of the nurse's testimony to be determined by the jury.\\\", \\\"inadmissible, as the x-rays were destroyed post-operation and do not pertain to the surgeon's performance during the surgery.\\\", \\\"inadmissible, unless other evidence can corroborate the nurse's testimony.\\\", 'admissible, only if the nurse can prove that the surgeon intentionally destroyed the x-rays.', \\\"inadmissible, because the nurse's testimony is hearsay.\\\"]\",\"question\":\"A surgeon performed an appendectomy on a patient in an operating room at the local hospital. The surgeon was assisted by a nurse who was assigned to the operation by the hospital. During the patient's early convalescence, he complained of pain not explicable as an ordinary post-operative symptom. On investigation, it turned out that the surgeon, who had bandaged him following the operation, had done so in such a manner as to constrict certain blood vessels. The faulty bandaging had caused acute pain and retarded the patient's recovery. After the patient's eventual recovery, he sued the surgeon for malpractice, claiming $25,000 in damages. In his case-in-chief, the patient called the nurse to testifS' that shortly after the surgery, she saw the surgeon destroy the postoperative x-rays of the patient's abdomen. Upon objection by the surgeon's attorney, the trial judge should rule the nurse's testimony\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}