{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcfa0de2-6795-5a43-b5a9-109f1cdc6c58","task_key":"default--test--6742","task_revision_id":"1","upstream_id":"6742","short_description":"Laryngoscopic examination of the vocal folds after suspected recurrent laryngeal…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Both cricothyroid and interarytenoid muscles are fully active.', 'Cricothyroid muscle is weak but the interarytenoids are paralyzed.', 'cricothyroid muscle is paralyzed but the interarytenoids are fully active.', 'cricothyroid muscle is paralyzed but the interarytenoids are weak.', 'Cricothyroid muscle is weak but the interarytenoids are fully active.', 'cricothyroid muscle is still functional but the interarytenoid muscles are fully active.', 'Interarytenoid muscles are still functional but the cricothyroid is weak.', 'Both cricothyroid and interarytenoid muscles are paralyzed.', 'cricothyroid muscle is still functional but the interarytenoids are weak.', 'Interarytenoid muscles are still functional but the cricothyroid is fully active.']\",\"question\":\"Laryngoscopic examination of the vocal folds after suspected recurrent laryngeal nerve damage usually shows that the vocal fold on the injured side is tensed and in an intermediate position between adduction and abduction. This is because\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}