{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4399e03e-ac0d-5f59-ad50-912dc138d72a","task_key":"default--test--2306","task_revision_id":"1","upstream_id":"2306","short_description":"Behavior disorders are commonly classified by professionals according to…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The classification system is updated too frequently, causing confusion among professionals.', 'The system is overly reliant on biological explanations, neglecting social and environmental factors.', 'The system does not adequately account for cultural and contextual differences in behavior.', \\\"The system's categories are overly broad, leading to misdiagnosis.\\\", 'The classification system is too complex', 'The criticisms include that the diagnostic classes are not homogeneous, the system is not reliable, and the system lacks validity.', 'The diagnostic criteria are considered too subjective, leading to inconsistent diagnoses.', 'The classification system lacks broad categories', 'The classification system is too simplistic and does not capture the full spectrum of human behavior.', 'The system relies too much on patient self-report']\",\"question\":\"Behavior disorders are commonly classified by professionals according to categories defined by the American Psychiatric Association. What are some criticisms of this classification system?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}