{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ba7d9e2-45f1-5742-a5db-290bb2d52e43","task_key":"all--test--5ba7d9e2-45f1-5742-a5db-290bb2d52e43","task_revision_id":"1","upstream_id":"","short_description":"An appliance store was using part of a public alley to unload its trucks. There…","config":"all","split":"test","body":"{\"choices\":\"['No, the plaintiff clearly assumed the risk by entering a dangerous loading/unloading area.', 'No, because the plaintiff in effect became a trespasser when he walked into a dangerous loading/unloading area.', 'Yes, because the plaintiff pedestrian had a right to be where he was and there were no signs telling him otherwise.', 'Yes, because the chain of causation was generally foreseeable in that it was a logical consequence of a truckload of appliances falling violently onto the road.']\",\"question\":\"An appliance store was using part of a public alley to unload its trucks. There were no warning signs keeping pedestrians out or warning of unloading trucks. A pedestrian walked through the general vicinity but he avoided the unloading area. One of the workers pulled the wrong lever and a load of commercial refrigerators crashed into the street, causing a rumbling tremor in the road. The tremor caused a heavy box to fall off of a parked truck that was about 30 yards away from the loading dock. It fell on the pedestrian, causing serious injury. The pedestrian sued the appliance store for negligence, and the store defended on the basis that it could not foresee such an accident and that the tremor and the box that fell were superseding acts. Will the pedestrian likely survive the store's motion to dismiss pedestrian's lawsuit for damages.\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}