{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e01cb98e-e6dd-5898-b22b-f092576b7b0d","task_key":"all--test--e01cb98e-e6dd-5898-b22b-f092576b7b0d","task_revision_id":"1","upstream_id":"","short_description":"In most states, the division of homicide into degrees is distinguished according…","config":"all","split":"test","body":"{\"choices\":\"[\\\"the causal relationship between the defendant's act and the resulting death.\\\", 'the attendant circumstances surrounding the death.', 'the nature of the act causing the death.', \\\"the defendant's state of mind at the time the killing was committed.\\\"]\",\"question\":\"In most states, the division of homicide into degrees is distinguished according to which of the following?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}