{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"535ead0f-7c97-5d56-aea5-3f5cc0572fbc","task_key":"all--test--535ead0f-7c97-5d56-aea5-3f5cc0572fbc","task_revision_id":"1","upstream_id":"","short_description":"The head coach of the local high school football team was also employed as a…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the player was injured while engaged in a school activity.', 'Yes, because under the circumstances the coach would be strictly liable.', 'No, because the coach was not negligent by continuing the practice.', 'No, because lightning is an act of God.']\",\"question\":\"The head coach of the local high school football team was also employed as a physical education teacher at the high school. It was a late autunm afternoon, and the football team was practicing for its upcoming game against its archrival. While the team was practicing, the skies darkened as a storm approached. Minutes later, it began to rain heavily. Even though the storm intensified, the coach did not stop the practice because he had no reason to believe the storm presented a danger to his players. Suddenly, a bolt of lightning struck and killed the team's star player. This jurisdiction has abolished governmental immunity, and a school board may be sued as a private entity. If the player's parents bring a wrongful death action against the coach and the high school board, will they prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}