{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4ddb255-9967-5676-ae5f-53efc51f20b5","task_key":"all--test--e4ddb255-9967-5676-ae5f-53efc51f20b5","task_revision_id":"1","upstream_id":"","short_description":"A woman and her neighbor had often talked about the need for a fence between…","config":"all","split":"test","body":"{\"choices\":\"['Damages for personal injuries cannot be recovered in a contract action.', 'Damages for personal injuries to the neighbor were not within the contemplation of the woman and the handyman at the time they entered into their agreement.', 'The neighbor is only an incidental beneficiary of the agreement between the woman and the handyman.', 'The neighbor has no standing to assert such an action, since he was not a party to the agreement between the woman and the handyman.']\",\"question\":\"A woman and her neighbor had often talked about the need for a fence between their properties. Finally, the woman hired a handyman to build a wooden stake fence between her property and her neighbor's property for $10,000. The handyman built the fence as agreed, and the woman paid him the sum of $10,000. Two weeks later, the neighbor was operating his lawn mower near the property line between his and the woman's property when a section of the fence suddenly fell on top of him, seriously injuring him. A subsequent investigation revealed that the handyman did not properly reinforce the fence. Which of the following is the handyman's best defense in a contract action brought against him by the neighbor, in which the only damages alleged are those for the neighbor's personal injuries?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}