{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6793b33b-a7bc-593b-a19f-3fd6a7dc2f99","task_key":"all--test--6793b33b-a7bc-593b-a19f-3fd6a7dc2f99","task_revision_id":"1","upstream_id":"","short_description":"A hot-air balloon touring company operated near a golf course. The company's…","config":"all","split":"test","body":"{\"choices\":\"['No, because the balloon was out of control when it struck the golfer.', 'No, because the company was engaged in an abnormally dangerous activity.', 'Yes, because the balloon landed to avoid crashing.', \\\"Yes, because the golfer assumed the risk by coming onto the company's property.\\\"]\",\"question\":\"A hot-air balloon touring company operated near a golf course. The company's property was separated from the golf course by a fence on which the company had posted signs warning people not to enter the property because of the dangers of balloons landing. A golfer on the golf course hit an errant shot onto the company's property, ignored the warning signs, and jumped over the fence to retrieve her golf ball. At about the same time, one of the company's balloons experienced mechanical problems and had to make an emergency landing to avoid crashing. The balloon, which was out of control when it landed, struck the golfer and injured her. The jurisdiction has decided that hot-air ballooning is an abnormally dangerous activity. In an action by the golfer against the company, does the company have any affirmative defenses?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}