{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"297c6e49-1606-5202-8b27-7a5d9470879e","task_key":"all--test--297c6e49-1606-5202-8b27-7a5d9470879e","task_revision_id":"1","upstream_id":"","short_description":"After weeks of deliberation, the defendant decided to rob a local liquor store.…","config":"all","split":"test","body":"{\"choices\":\"['Felony murder.', 'Involuntary manslaughter.', 'Voluntary manslaughter.', 'No crime.']\",\"question\":\"After weeks of deliberation, the defendant decided to rob a local liquor store. The defendant purchased a ski mask and then went into the liquor store, put his gun into the face of the clerk, and demanded the money from the register. Terrified, the clerk gave him the money from the register. The defendant pocketed the money, grabbed a bottle of tequila off the shelf for his celebration, and then ran back to the car and drove off. About an hour later, the defendant was slowly approaching his hideout in the outskirts of town when a young child suddenly darted in front of the defendant's car. He applied the brakes but couldn't stop in time. The car struck the child, killing her. For the child's death, the defendant should be found guilty of which, if any, of the following crimes?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}