{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6295d8d-8e6b-5d90-97aa-45e3844473a0","task_key":"default--test--6151","task_revision_id":"1","upstream_id":"6151","short_description":"Excess disability refers to symptoms that are","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Not due to the disease but due instead to malignant social psychology', 'Those that are experienced only by individuals over a certain age', 'In excess of the average for that particular disorder', 'Symptoms that occur as a side effect of treatment for the disease', 'Symptoms not related to any disease but are due to a mental disorder', 'Symptoms that are exacerbated by comorbid conditions', 'Those that appear only after the disease has been cured', 'Those that only appear in chronic diseases', 'Those that can be successfully treated by medicine if they are caught early enough', \\\"Found only in the last stage of Alzheimer's\\\"]\",\"question\":\"Excess disability refers to symptoms that are\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}