{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"09e923da-a831-5fe3-82d3-d6e9b8cea91f","task_key":"all--test--09e923da-a831-5fe3-82d3-d6e9b8cea91f","task_revision_id":"1","upstream_id":"","short_description":"A defendant was walking through a park when he decided to rob someone. He hid…","config":"all","split":"test","body":"{\"choices\":\"['Murder.', 'Felony murder.', 'Involuntary manslaughter.', 'Voluntary manslaughter.']\",\"question\":\"A defendant was walking through a park when he decided to rob someone. He hid behind a tree, lying in wait for a victim to approach. Shortly thereafter, a girl was strolling in the park when the defendant suddenly jumped from his hiding place and accosted her. Although the defendant intended only to rob his victim, he punched her in the mouth and she fell to the ground. The defendant then grabbed her purse and fled. Unknown to the defendant, the girl suffered a fractured skull when her head struck the pavement. She subsequently died from her head injuries. Which of the following is the most serious crime for which the defendant can be found guilty?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}