{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a00ed10a-a8f2-5ff4-9228-1ea6c0062bb2","task_key":"all--test--a00ed10a-a8f2-5ff4-9228-1ea6c0062bb2","task_revision_id":"1","upstream_id":"","short_description":"A 26-year-old graduate student knew that a classmate was a staunch supporter of…","config":"all","split":"test","body":"{\"choices\":\"[\\\"the classmate's use of the brass knuckles exceeded consent under the circumstances.\\\", 'the classmate was only acting in self-defense.', \\\"the graduate student's insults provoked the classmate's use of excessive force.\\\", \\\"the classmate's punch would have inflicted the same harm with or without the use of the brass knuckles.\\\"]\",\"question\":\"A 26-year-old graduate student knew that a classmate was a staunch supporter of animal rights. As they were leaving class one afternoon, the graduate student walked beside the classmate and chanted, \\\"Animal experimentation is great. \\\" The classmate angrily confronted the graduate student and said, \\\"Let's settle this outside in a fist fight. \\\" The graduate student replied, \\\"You're on. \\\"The two students went outside and \\\"squared off. \\\" The graduate student threw the first punch and hit the classmate in the face with his bare fist. Unknown to the graduate student, the classmate placed a set of brass knuckles on his fist and hit the graduate student in the face with them. The force of the blow broke the graduate student's nose. It was later determined that the use of the brass knuckles inflicted exactly the same damage as if the classmate had hit the graduate student with his bare fist. The best argument for rejecting the defense of consent in an action by the graduate student for either assault or battery is that\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}