{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e8ed2a7-af3d-5e7c-951b-e8fbf109b379","task_key":"all--test--7e8ed2a7-af3d-5e7c-951b-e8fbf109b379","task_revision_id":"1","upstream_id":"","short_description":"The owner of a drug store hired an independent contractor to renovate the store.…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because the glass door was defective.', 'prevail, because the sales representative represented the plate glass as being \\\"super safe. \\\"', 'not prevail, because the contractor negligently installed the plate glass door.', 'not prevail, because the son negligently pushed open the glass door.']\",\"question\":\"The owner of a drug store hired an independent contractor to renovate the store. Among other things, the renovations included installing a new front door made of plate glass. The contractor purchased the door from a glass door company. The sales representative of the glass door company told the contractor and the owner that plate glass was \\\"super safe\\\" and could withstand considerable pressure. The contractor installed the door, completed the other renovation work, and the store reopened for business later that same month. A few weeks later, a shopper and her 9-year- old son went to the drug store to purchase some merchandise. As they approached the store, the son was skipping ahead of his mother. Until he was within a few steps of the door, he thought it was open. When he realized it was, in fact, closed, he slowed down and, while still \\\"going faster than walking,\\\" pushed his hand against the handle to push open the door. His hand slipped off the handle and, at the same time, there was a \\\"kind of exploding noise. \\\" Due to a defect, the glass shattered and broke, cutting the son's hand and wrist. A subsequent inspection revealed that the glass door had been negligently installed by the contractor. If the shopper, on behalf of her son, asserts a claim against the glass door company based on strict products liability, the plaintiff will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}