{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57138e38-0862-5ae8-bc59-a7fde539d081","task_key":"default--test--6483","task_revision_id":"1","upstream_id":"6483","short_description":"The coronal suture joins the","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['parietal, squamous temporal and greater wing of the sphenoid.', 'left and right parietal bones.', 'parietal and sphenoid bones.', 'frontal and occipital bones.', 'squamous temporal and greater wing of the sphenoid.', 'frontal and parietal bones.', 'parietal and occipital bones.', 'parietal and ethmoid bones.', 'left and right temporal bones.', 'parietal and squamous temporal bones.']\",\"question\":\"The coronal suture joins the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}