{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5e9e83c-a4aa-59da-b37a-441ba4f2b7b4","task_key":"all--test--e5e9e83c-a4aa-59da-b37a-441ba4f2b7b4","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is a method of torture that Dershowitz believes is…","config":"all","split":"test","body":"{\"choices\":\"[\\\"having a sterilized needle shoved under the victim's fingernails\\\", 'electrocution at nonlethal levels', 'submerging the victim in water so he or she feels as if he or she is drowning', 'none of the above; Dershowitz thinks torture is wrong in all cases.']\",\"question\":\" Which of the following is a method of torture that Dershowitz believes is appropriate in those cases where torture is permissible?\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}