{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"08bd34c4-5d8c-521a-bd6b-601f92e0f0d6","task_key":"all--test--08bd34c4-5d8c-521a-bd6b-601f92e0f0d6","task_revision_id":"1","upstream_id":"","short_description":"In response to the argument that the death penalty does not deter murderers, van…","config":"all","split":"test","body":"{\"choices\":\"['he is still confident that the threat of the death penalty at least sometimes prevents murder.', 'criminal law is supposed to protect the innocent over murderers.', 'the severity and finality of the death penalty is appropriate to the seriousness and finality of murder.', 'all of the above']\",\"question\":\" In response to the argument that the death penalty does not deter murderers, van den Haag claims that\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}