{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c2172e6-04d7-5d62-a4cc-2e0baf1903b8","task_key":"default--test--6027","task_revision_id":"1","upstream_id":"6027","short_description":"The muscles of the soft palate are innervated by branches of the","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['facial and vagus nerves.', 'trigeminal and hypoglossal nerves.', 'facial and glossopharyngeal nerves.', 'trigeminal and glossopharyngeal nerves.', 'optic and oculomotor nerves.', 'olfactory and optic nerves.', 'facial and hypoglossal nerves.', 'glossopharyngeal and vagus nerves.', 'trigeminal and facial nerves.', 'trigeminal and vagus nerves.']\",\"question\":\"The muscles of the soft palate are innervated by branches of the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}