{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee27ba78-3645-5723-b82d-d5fc4479a017","task_key":"default--test--1362","task_revision_id":"1","upstream_id":"1362","short_description":"A football player was the star fulllack for the local college football team.…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['not prevail, because the coach had the right to express his opinion.', \\\"prevail, because the coach's remarks led directly to the football player's emotional distress and subsequent suicide attempt.\\\", 'prevail, because the coach intended to cause him to suffer emotional distress.', \\\"not prevail, because the football player's drug overdose resulted from his own voluntary act.\\\", \\\"prevail, because the coach's remarks constituted bullying behavior.\\\", \\\"prevail, because the coach's remarks were intended to inflict emotional distress and resulted in the football player's suicide attempt.\\\", 'not prevail, because the coach acted reasonably under the circumstances, since everyone on the team hated the football player.', \\\"prevail, because the coach's remark did, in fact, cause the football player to suffer emotional distress.\\\", \\\"not prevail, because the coach's remarks do not meet the legal standard for intentional infliction of emotional distress.\\\", \\\"not prevail, because the football player's overdose was an unforeseen consequence of the coach's remarks.\\\"]\",\"question\":\"A football player was the star fulllack for the local college football team. After missing two practices, the football player was dropped from the team by the head football coach. Following his dismissal, the football player met with the coach and asked if he could rejoin the team. The coach said that the football player was despised by the other players and under no circumstances could he return to the team. As the football player was leaving the coach's office, feeling very dejected, the coach then said to him, \\\"Hope you decide to transfer, because everybody hates your guts around here. \\\"Later that same evening, the football player wrote a suicide note in which he stated, \\\"The coach is responsible for my despondency. If I can't play football, I don't want to live. \\\" After swallowing a bottle of Quaalude barbiturates, the football player fell unconscious in his dormitory room. Moments later, the football player's roommate entered the room and saw his limp body on the floor. The roommate read the suicide note and then attempted to administer aid. Failing to revive him, the roommate picked up the football player and carried him to the college's first aid center. The football player received prompt medical attention and soon recovered from his drug overdose. If the football player asserts a claim against the coach based on intentional infliction of emotional distress, the football player will most likely\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}