{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6bac9dc-fcad-55ac-a3d8-5b15bc763411","task_key":"all--test--d6bac9dc-fcad-55ac-a3d8-5b15bc763411","task_revision_id":"1","upstream_id":"","short_description":"A homeowner, his daughter, and his sister are the owners of three contiguous…","config":"all","split":"test","body":"{\"choices\":\"['the sister must remove the wall because she has no right to obstruct the flow of such surface water.', \\\"the sister must remove the wall at the daughter's expense.\\\", 'the sister may leave the wall without being liable to the daughter for money damages.', 'the sister may leave the wall, but she will be liable to the daughter for money damages.']\",\"question\":\"A homeowner, his daughter, and his sister are the owners of three contiguous lots in the city. A downward slope exists from the homeowner's land to the sister's land. The homeowner's and daughter's lots were in an unimproved natural state. The sister, however, had lived for 10 years in a house that she had built on her property. The daughter, in preparation for building a house on her lot, carefully excavated an area eight feet deep for the purpose of building a basement. The daughter completed construction of her house and macadamized an area for use as a driveway without changing the former contours of the land. Shortly thereafter, the sister began to make complaints to the daughter about the flooding of her basement, which she claimed had been previously free of water. The sister then built a concrete wall three feet along her border with the daughter to prevent the flow of rain water running onto her land from the daughter's property. This caused the surface water to stand and become stagnant on the daughter's land. The daughter demanded that the sister remove the wall, and upon the sister's refusal, the daughter brought an appropriate action to compel removal. The most likely result is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}