{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"453412fe-bfa9-5cc4-80b1-71d71e490bb5","task_key":"all--test--453412fe-bfa9-5cc4-80b1-71d71e490bb5","task_revision_id":"1","upstream_id":"","short_description":"A defendant had an argument with his neighbor. As they were quarreling, the…","config":"all","split":"test","body":"{\"choices\":\"['Murder.', 'Voluntary manslaughter.', 'Involuntary manslaughter.', 'Battery.']\",\"question\":\"A defendant had an argument with his neighbor. As they were quarreling, the defendant pulled out his penknife intending only to frighten the neighbor. The defendant accidentally slightly nicked the neighbor's arm with the knife. Unknown to the defendant, his neighbor was a hemophiliac who then died from the cut. What is the most serious crime that the defendant can be convicted of?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}