{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8119d76f-cb5f-548e-9bdc-d897b8407d2a","task_key":"default--test--2388","task_revision_id":"1","upstream_id":"2388","short_description":"A research project compared the performance of learning disabled and of normal…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['can account for more variables simultaneously', 'is a more recent development in statistical analysis', 'reduces the need for subsequent regression analysis', 'can handle larger datasets more effectively', 'usually offers greater protection against Type Il error', 'requires less computational power', 'usually offers greater protection against a Type I error', 'allows for more detailed individual participant analysis', 'is conceptually and computationally a simpler and more efficient approach']\",\"question\":\"A research project compared the performance of learning disabled and of normal students on ten scales of the Wechsler Intelligence Scale for Children ~ Revised (WISC-R). Ten univariate analyses of variance (ANOVAs) were run to test for differences between the two groups, but use of a multivariate analysis of variance (MANOVA) would have been more desirable because MANOVA\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}