{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc932470-34f9-589d-a530-329f0df3bd3a","task_key":"all--test--cc932470-34f9-589d-a530-329f0df3bd3a","task_revision_id":"1","upstream_id":"","short_description":"A buyer purchased a shovel at a local hardware store. The shovel had a metal…","config":"all","split":"test","body":"{\"choices\":\"['The buyer, because the manufacturer conceded that the shovel was defective.', 'The buyer, because the manufactuter failed to exercise reasonable care in the production and inspection of its product.', 'The manufacturer, because the manufacturer did not guarantee that the shovel would never break.', \\\"The manufacturer, because the buyer's negligence in the way he used the shovel was one of the causes of his injury.\\\"]\",\"question\":\"A buyer purchased a shovel at a local hardware store. The shovel had a metal blade and a wooden handle. While the buyer was using the shovel to dig out a tree stump in his back yard, the handle broke in half, causing the buyer to fall and injure his back. The buyer brought a negligence claim against the manufacturer of the shovel. He proved that the shovel broke because of a manufacturing defect that made the handle too weak and that the manufacturer acted unreasonably in allowing this to happen. While conceding that the handle had a defect that caused it to break, the manufacturer established that if the buyer had been using the shovel carefully, he would not have fallen down and would not have been injured when the shovel broke. Assume that the case is governed by the law of a jurisdiction that follows traditional contributory negligence rules. Who is most likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}