{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"778df4d9-b9e5-5698-bf39-c388cb901dd1","task_key":"all--test--778df4d9-b9e5-5698-bf39-c388cb901dd1","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is a problem for Kant's principle of equality and an \"eye…","config":"all","split":"test","body":"{\"choices\":\"['It fails to tell us what the appropriate punishment would be for all cases of criminal action.', 'It does not provide an adequate measure of moral desert.', 'It requires that we punish criminals in ways that are morally unacceptable.', 'all of the above']\",\"question\":\" Which of the following is a problem for Kant's principle of equality and an \\\"eye for an eye\\\" according to Nathanson?\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}