{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4b10a3f-5cad-599c-b70e-e91b75e66d4e","task_key":"default--test--6032","task_revision_id":"1","upstream_id":"6032","short_description":"A patient with damage to their cervical sympathetic chain will show the…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Pupillary constriction and vasodilation of facial vessels', 'Pupillary dilation', 'Pupillary constriction, vasoconstriction of facial vessels and increased facial sweating', 'Pupillary dilation, vasoconstriction of facial vessels, decreased facial sweating and increased lacrimation', 'Pupillary constriction, vasodilation of facial vessels, increased facial sweating and decreased lacrimation', 'Pupillary constriction, vasodilation of facial vessels and increased facial sweating', 'Pupillary constriction and vasodilation of facial vessels, decreased facial sweating and increased lacrimation', 'Pupillary constriction', 'Pupillary dilation and vasoconstriction of facial vessels', 'Pupillary dilation, vasoconstriction of facial vessels and decreased facial sweating']\",\"question\":\"A patient with damage to their cervical sympathetic chain will show the following signs and symptoms.\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}