{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"163bda1d-0aa7-5df6-9489-148e061353e2","task_key":"default--test--6482","task_revision_id":"1","upstream_id":"6482","short_description":"The mucosa of the upper larynx is innervated by the","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['recurrent laryngeal nerve which is the afferent limb of the cough reflex.', 'internal laryngeal nerve which is the afferent limb of the gag reflex.', 'vagus nerve which is the afferent limb of the gag reflex.', 'vagus nerve which is the afferent limb of the cough reflex.', 'internal laryngeal nerve which is the afferent limb of the cough reflex.', 'superior laryngeal nerve which is the afferent limb of the gag reflex.', 'external laryngeal nerve which is the afferent limb of the cough reflex.', 'external laryngeal nerve which is the afferent limb of the gag reflex.', 'recurrent laryngeal nerve which is the afferent limb of the gag reflex.', 'superior laryngeal nerve which is the afferent limb of the cough reflex.']\",\"question\":\"The mucosa of the upper larynx is innervated by the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}