{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"639c9278-c90a-5b1c-a451-32503048743d","task_key":"default--test--6031","task_revision_id":"1","upstream_id":"6031","short_description":"Which of the following foramina pierce the sphenoid bone?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The foramen ovale', 'The foramen ovale, rotundum and spinosum', 'The foramen ovale, foramen spinosum and foramen lacerum', 'The foramen ovale and rotundum', 'The foramen ovale, rotundum, spinosum and superior orbital fissure', 'The foramen ovale, rotundum, spinosum and carotid canal', 'Only the foramen rotundum', 'The foramen ovale, rotundum and spinosum and foramen lacerum', 'The foramen ovale, rotundum, spinosum and foramen magnum', 'The foramen ovale and foramen spinosum']\",\"question\":\"Which of the following foramina pierce the sphenoid bone?\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}