{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4262de5e-3c52-5902-8331-44d315d8f7b7","task_key":"all--test--4262de5e-3c52-5902-8331-44d315d8f7b7","task_revision_id":"1","upstream_id":"","short_description":"A center and a quarterback were 14-year-old eighth graders at the local high…","config":"all","split":"test","body":"{\"choices\":\"['be held responsible, because he committed a battery by hitting the center in the chest.', \\\"be held responsible, because the center's consent was ineffective.\\\", \\\"not be held responsible, because the center's consent was effective.\\\", 'not be held responsible, because the quarterback delivered the punch with greater strength than the center anticipated.']\",\"question\":\"A center and a quarterback were 14-year-old eighth graders at the local high school. They were both members of the school's junior varsity football team. The center weighed 170 pounds and was stocky in build. The quarterback was lanky and weighed about 145 pounds. The center and the quarterback often engaged in friendly tests of strength, such as arm wrestling and weight lifting contests, to see who was stronger. One afternoon, both youngsters were in the school gymnasium getting ready for football practice. As they were putting on their football uniforms, the center turned to the quarterback and said, \\\"Hey, weakling, why don't you punch me in the chest with your best shot?\\\" The quarterback replied, \\\"No, I'm afraid I might hurt you. \\\" The center then said, \\\"Are you kidding? You can't hurt me. C'mon, give me your best shot. \\\" The center stood up, stuck out his muscular chest, and prepared to have the quarterback hit him. The quarterback proceeded to cock his fist and then punched the center in the chest as hard as he could. Immediately thereafter, the center slumped to the floor, gasping for air. Realizing that the center was seriously hurt, the quarterback tried to render assistance but to no avail. Seconds later the center stopped breathing and died. Unknown to either the center or the quarterback, the center had a defective heart and suffered a heart attack as a result of the blow to the chest. In a wrongful death action, the quarterback will likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}