{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"31fd92dc-510e-5bda-8a4f-46016e3b0d72","task_key":"all--test--31fd92dc-510e-5bda-8a4f-46016e3b0d72","task_revision_id":"1","upstream_id":"","short_description":"A football player was the star fulllack for the local college football team.…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because the coach intended to cause him to suffer emotional distress.', \\\"prevail, because the coach's remark did, in fact, cause the football player to suffer emotional distress.\\\", \\\"not prevail, because the football player's drug overdose resulted from his own voluntary act.\\\", 'not prevail, because the coach acted reasonably under the circumstances, since everyone on the team hated the football player.']\",\"question\":\"A football player was the star fulllack for the local college football team. After missing two practices, the football player was dropped from the team by the head football coach. Following his dismissal, the football player met with the coach and asked if he could rejoin the team. The coach said that the football player was despised by the other players and under no circumstances could he return to the team. As the football player was leaving the coach's office, feeling very dejected, the coach then said to him, \\\"Hope you decide to transfer, because everybody hates your guts around here. \\\"Later that same evening, the football player wrote a suicide note in which he stated, \\\"The coach is responsible for my despondency. If I can't play football, I don't want to live. \\\" After swallowing a bottle of Quaalude barbiturates, the football player fell unconscious in his dormitory room. Moments later, the football player's roommate entered the room and saw his limp body on the floor. The roommate read the suicide note and then attempted to administer aid. Failing to revive him, the roommate picked up the football player and carried him to the college's first aid center. The football player received prompt medical attention and soon recovered from his drug overdose. If the football player asserts a claim against the coach based on intentional infliction of emotional distress, the football player will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}