{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2328d767-a602-5d33-8817-6e36a3612dd8","task_key":"all--test--2328d767-a602-5d33-8817-6e36a3612dd8","task_revision_id":"1","upstream_id":"","short_description":"On Tuesday, a homeowner purchased a new riding lawnmower from a department…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Yes, because the lawnmower entered onto the neighbor's property.\\\", 'Yes, because the homeowner is strictly liable for the damage caused by the defective mower.', 'No, because the homeowner was using the mower for its intended purpose.', 'No, because the defective motor switch was not discoverable by reasonable inspection.']\",\"question\":\"On Tuesday, a homeowner purchased a new riding lawnmower from a department store. The next day, he was cutting the grass in his back yard with the mower when it started to rain. He stopped the mower and turned off the motor switch. He went inside and decided to wait until the rain stopped before mowing the rest of the lawn. A few minutes later while the homeowner was inside his house, the lawnmower suddenly lurched forward, rolled down a hill, and entered the property of his neighbor. The mower cut the neighbor's prize rosebushes, destroying them. It was later determined that the lawnmower's motor switch was defective. Even though the homeowner had turned the starter switch to the off position, the engine did not shut off. Because the mower was built with a new silent rotary engine, the homeowner had no reason to know that it was still running when he went into the house. The homeowner would not have been able to discover the defect by any feasible means of inspection. If the neighbor asserts a claim against the homeowner for trespass, will the neighbor prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}