{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a51080bd-ee25-5005-9b8c-aeb0a527a2f6","task_key":"default--test--1677","task_revision_id":"1","upstream_id":"1677","short_description":"An appliance store was using part of a public alley to unload its trucks. There…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, the accident was caused by a worker's mistake, not a lack of warning signs or barriers.\\\", 'No, because the plaintiff in effect became a trespasser when he walked into a dangerous loading/unloading area.', 'Yes, the appliance store has a duty of care to ensure the safety of pedestrians around their loading/unloading area.', 'Yes, the appliance store was negligent in not having the proper warning signs or barriers to prevent such accidents.', \\\"Yes, the store's negligence in operating their loading dock caused a dangerous situation that led to the pedestrian's injury.\\\", 'No, because the pedestrian was beyond the immediate danger area and the store could not have predicted the box falling from a separate truck.', 'No, the plaintiff clearly assumed the risk by entering a dangerous loading/unloading area.', 'Yes, because the plaintiff pedestrian had a right to be where he was and there were no signs telling him otherwise.', \\\"No, the accident was a result of a series of unforeseeable events and not directly due to the store's actions.\\\", 'Yes, because the chain of causation was generally foreseeable in that it was a logical consequence of a truckload of appliances falling violently onto the road.']\",\"question\":\"An appliance store was using part of a public alley to unload its trucks. There were no warning signs keeping pedestrians out or warning of unloading trucks. A pedestrian walked through the general vicinity but he avoided the unloading area. One of the workers pulled the wrong lever and a load of commercial refrigerators crashed into the street, causing a rumbling tremor in the road. The tremor caused a heavy box to fall off of a parked truck that was about 30 yards away from the loading dock. It fell on the pedestrian, causing serious injury. The pedestrian sued the appliance store for negligence, and the store defended on the basis that it could not foresee such an accident and that the tremor and the box that fell were superseding acts. Will the pedestrian likely survive the store's motion to dismiss pedestrian's lawsuit for damages.\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}