{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28a53f39-286a-59f2-8253-9325d5a79b50","task_key":"all--test--28a53f39-286a-59f2-8253-9325d5a79b50","task_revision_id":"1","upstream_id":"","short_description":"A musician owned a summer cottage on the lake. A neighbor, an adjoining land…","config":"all","split":"test","body":"{\"choices\":\"['be successful if he can prove that the neighbor was negligent in his excavations.', 'be successful, because the neighbor would be strictly liable for his removal of lateral support.', 'not be successful, because the neighbor could not be held liable for damage to an artificial structure such as a storage building.', \\\"not be successful, because an adjacent land owner is under no affirmative duty to laterally support his neighbor's land.\\\"]\",\"question\":\"A musician owned a summer cottage on the lake. A neighbor, an adjoining land owner, started a sand business, which required him to do heavy hauling of sand, rocks, dirt, and other materials on his property. The neighbor's excessive excavating in the course of conducting his sand removal business causes the collapse of a large storage building on the musician's property. If the musician brings an action to recover damages for the collapse of his storage building, the musician will probably\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}