{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fa1b4df-2452-56c5-8586-00b9b0ad9031","task_key":"all--test--0fa1b4df-2452-56c5-8586-00b9b0ad9031","task_revision_id":"1","upstream_id":"","short_description":"On Friday night, a driver was injured in an automobile accident after his…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because the stirrup broke while the traction apparatus was in normal use.', \\\"prevail, because the manufacturer's failure to include a safety latch made the traction apparatus defective.\\\", 'not prevail, because the traction apparatus was no longer within the control of the manufacturer.', 'not prevail, if the physician did not properly attach the stirrup when assembling the traction apparatus.']\",\"question\":\"On Friday night, a driver was injured in an automobile accident after his vehicle was sideswiped by an unidentified motorist. Following the accident, the driver was admitted to the hospital where he was treated for his injuries. While at the hospital, the driver was diagnosed with a broken back. He was put in a body cast and then placed in traction. The driver's physician assembled the traction apparatus and positioned the driver's left leg in an overhead stirrup at a 40-degree angle. The driver was instructed to lie on his back on the bed. On Saturday morning after breakfast, the driver requested a bedpan from a nurse. She was placing the bedpan under the driver's body when the stirrup, which was holding his leg, broke. This caused the driver's leg to fall against the metal framing along the side of the bed, fracturing his tibia. The traction apparatus was defective because the manufacturer had not included a safety latch on the stirrup device to prevent its falling off if not securely fastened in place. If the driver asserts a claim against the manufacturer, he will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}