{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49b91d41-a441-531b-b2fe-0fa1366ceda8","task_key":"default--test--6170","task_revision_id":"1","upstream_id":"6170","short_description":"Macrostomia results from failure of fusion of","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['left and right maxillary processes.', 'mandibular and hyoid arches.', 'maxillary and frontonasal processes.', 'mandibular and frontonasal processes.', 'frontonasal and hyoid arches.', 'frontonasal and maxillary processes.', 'left and right hyoid arches.', 'maxillary and hyoid arches.', 'left and right mandibular processes.', 'maxillary and mandibular processes.']\",\"question\":\"Macrostomia results from failure of fusion of\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}