{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"398d4820-7940-532f-b201-85d79154c6ca","task_key":"default--test--2702","task_revision_id":"1","upstream_id":"2702","short_description":"When the corpus callosum is cut in a right-handed patient, often the patient","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Cannot name objects placed in the right hand if the patient can see them', 'cannot recognize objects placed in the right hand', 'cannot name objects placed in the left hand if the patient cannot see them', 'cannot carry out well-learned motor tasks with the left hand', 'shows a decline in verbal intelligence', 'loses the ability to speak', 'Cannot name objects placed in the right hand if the patient cannot see them', 'cannot carry out well-learned motor tasks with the right hand', 'shows a decline in spatial intelligence', 'can no longer write with their right hand']\",\"question\":\"When the corpus callosum is cut in a right-handed patient, often the patient\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}