{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0302a31a-6f59-550a-90f9-6a07c5bb84ac","task_key":"default--test--1109","task_revision_id":"1","upstream_id":"1109","short_description":"A patient in a hospital was placed in a wheelchair with his broken leg extended…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the patient has not provided any evidence to prove that the hospital was responsible for the malfunctioning doors.', 'No, because the hospital has a duty to ensure the safety of all its patients and failed to do so in this case.', 'Yes, because the hospital was not aware of any defects in the doors.', 'Yes, because proof of an accident, by itself, does not establish that an injured person was a victim of negligence.', \\\"Yes, because the nurse's action was a superseding cause of the injury. \\\\x80\\\\x83\\\", 'No, because a jury could find that there was a latent defect in the doors.', 'No, because a jury could find the hospital liable for negligence based on res ipsa loquitur.']\",\"question\":\"A patient in a hospital was placed in a wheelchair with his broken leg extended straight out in front of him. As a nurse employed by the hospital was pushing the wheelchair through a set of automatic doors at a normal pace, the doors closed on the patient's foot, injuring it. The nurse attempted to pull the wheelchair back through the doors. This action caused the doors to close more tightly on the patient's foot, injuring it further. The patient sued the hospital, alleging improper maintenance of the doors. The patient has produced no evidence of specific conduct or neglect on the part of the hospital that would have caused the automatic doors to malfunction. The hospital has moved for summary judgment. Should the court grant the hospital's motion?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}