{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9c3d940-326b-5afd-ba94-4850b1d4e69f","task_key":"default--test--873","task_revision_id":"1","upstream_id":"873","short_description":"A patient who had surgery to repair a rupture in her spleen followed up with a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['She does not have to produce the MRI scan because her testimony is enough to prove negligence.', 'She must produce the MRI scan as it is a tangible evidence and can be examined by the court.', 'She must produce the MRI scan because the jury needs to see the physical evidence.', \\\"She does not have to produce the MRI scan because it is the hospital's responsibility to provide the evidence.\\\", 'She must produce the MRI records because it represents the best evidence under the best evidence rule.', 'She does not have to produce the MRI scan because she can testify as to the content of her own medical records.', 'She does have to produce the MRI scan because a layperson cannot testify to personal observations.', 'She does not have to produce the MRI scan because a layperson can testify to things of common knowledge, such as what a sponge looks like.', 'She does not have to produce the MRI scan because the hospital already admitted to leaving the sponges in her body.', 'She must produce the MRI scan because only a medical professional can interpret the results.']\",\"question\":\"A patient who had surgery to repair a rupture in her spleen followed up with a full body MRI due to excruciating pain that had developed. During the MRI, the technicians saw at least two identifiable sponges that had been left in her body. She had to be hospitalized for additional surgery. The patient sued the hospital and doctor for medical malpractice. She did not produce the printouts of the MRI. Instead, she attempted to testify to what she observed on the MRI scan. The defense objected and insisted that she must produce the actual MRI printouts. What is the most likely ruling of the trial judge with respect to the objection?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}