{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"817c30c1-257a-5ac7-b686-16b02d2949e2","task_key":"default--test--6301","task_revision_id":"1","upstream_id":"6301","short_description":"A pulse may be felt on a patient's face where the","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['superficial temporal artery crosses the zygomatic arch.', 'superficial temporal artery crosses the mandible.', 'maxillary artery crosses the mandible.', 'superficial temporal artery crosses the orbital rim.', 'facial artery crosses the orbital rim.', 'superficial temporal artery crosses the maxilla.', 'facial artery crosses the zygomatic arch.', 'maxillary artery crosses the orbital rim.', 'facial artery crosses the maxilla.', 'facial artery crosses the mandible.']\",\"question\":\"A pulse may be felt on a patient's face where the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}