{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e70b820-8d3a-548a-8d0e-14e2eba7af7d","task_key":"default--test--6305","task_revision_id":"1","upstream_id":"6305","short_description":"The dorsal roots of all spinal nerves contain","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['motor and autonomic neuronal processes.', 'both autonomic and motor neuronal processes.', 'both sensory and motor neuronal processes.', 'sensory and autonomic neuronal processes.', 'motor neuronal processes.', 'only autonomic neuronal processes.', 'no neuronal processes.', 'sensory, motor and autonomic neuronal processes.', 'sensory neuronal processes.', 'autonomic neuronal processes.']\",\"question\":\"The dorsal roots of all spinal nerves contain\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}