{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40879b7f-7085-53ec-8f2b-4d159663b3f3","task_key":"default--test--996","task_revision_id":"1","upstream_id":"996","short_description":"A homeowner resides downhill from a metal fabrication facility. She has sued…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the solvent supplier should have taken precautions to prevent spills.', 'No, because the solvent supplier is directly responsible for the contamination.', \\\"Yes, because the solvent supplier can't be held accountable for the negligence of the facility owner.\\\", 'Yes, because there is no basis for allocating damages against the solvent supplier.', 'Yes, because the negligence of the facility owner is the primary cause of contamination.', 'No, because the solvent supplier is vicariously liable for damage inflicted by the facility owner.', 'Yes, because there is no evidence that the facility owner and the solvent supplier acted in concert.', 'Yes, because the homeowner does not have substantial proof of contamination.', 'No, because concurrent tortfeasors are jointly and severally liable for an indivisible injury.', 'No, because the solvent supplier and the facility owner are both responsible for the contamination, regardless of the amount each contributed.']\",\"question\":\"A homeowner resides downhill from a metal fabrication facility. She has sued both the owner of the facility and the supplier of a solvent used at the facility. She contends that contaminants, consisting mostly of the solvent, were released into the ground at the facility and have migrated and continue to migrate to her property, contaminating the soil, the groundwater, and her well. She alleges various acts of negligence on the part of the facility owner in causing the release of the contaminants into the ground. She also alleges that employees of the solvent supplier were negligent in frequently the solvent supplier were negligent in frequently spilling some of the solvent onto the ground while filling a rooftop tank at the facility. The solvent supplier has moved for summary judgment, arguing that if there was any contamination, the facility owner and the supplier independently contributed indeterminate amounts to the contamination and that therefore the homeowner cannot show how much damage each has inflicted on her. There is no evidence that the facility owner and the solvent supplier acted in concert. Should the court grant the summary judgment motion?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}