{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00cc88a2-70fa-51a1-a979-2487cbc1301a","task_key":"all--test--00cc88a2-70fa-51a1-a979-2487cbc1301a","task_revision_id":"1","upstream_id":"","short_description":"Defendant was waiting in line for post-Thanksgiving day sales. The door opened…","config":"all","split":"test","body":"{\"choices\":\"['First degree murder', 'Second degree murder', 'Involuntary manslaughter', 'voluntary manslaughter']\",\"question\":\"Defendant was waiting in line for post-Thanksgiving day sales. The door opened at 6 a.m. and hundreds of people rushed the entrance for a chance at a hugely discounted holiday gift. When defendant saw that there was a signed copy of a book written by his mother's favorite author, he jumped towards it, only to be blocked by a little old lady with a baseball bat. She was using the bat as a blocking tool, but he was determined to get the book, so he deliberately stuck his leg out in front of her and tripped her. She fell hard and in the process ended up with a blood clot in her leg, which traveled to her heart over a period of weeks, and ultimately caused her death. What crime can the defendant likely be convicted of?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}