{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"885513c7-6d92-5bc4-9d6d-170dd5d6e15e","task_key":"all--test--885513c7-6d92-5bc4-9d6d-170dd5d6e15e","task_revision_id":"1","upstream_id":"","short_description":"Laryngoscopic examination of the vocal folds after suspected recurrent laryngeal…","config":"all","split":"test","body":"{\"choices\":\"['cricothyroid muscle is still functional but the interarytenoid muscles are fully active.', 'cricothyroid muscle is still functional but the interarytenoids are weak.', 'cricothyroid muscle is paralyzed but the interarytenoids are fully active.', 'cricothyroid muscle is paralyzed but the interarytenoids are weak.']\",\"question\":\"Laryngoscopic examination of the vocal folds after suspected recurrent laryngeal nerve damage usually shows that the vocal fold on the injured side is tensed and in an intermediate position between adduction and abduction. This is because\",\"subject\":\"anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}