{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38c7631f-8fa9-533e-8df4-917a175f7be9","task_key":"all--test--38c7631f-8fa9-533e-8df4-917a175f7be9","task_revision_id":"1","upstream_id":"","short_description":"West argues that feminist rhetoric has","config":"all","split":"test","body":"{\"choices\":\"['obscures the harms of noncoerced, consensual sex by implying that because what is coerced is bad what is noncoerced is good.', 'obscures the harms of rape by equating all sex with nonconsensual sex.', 'by not acknowledging a distinction between what is good and what noncoerced, has led to the acknowledgement of the harms of consensual sex.', 'All of the above']\",\"question\":\" West argues that feminist rhetoric has\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}