{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f95e90b1-c9f5-530f-ba61-2088c1e22b21","task_key":"default--test--2391","task_revision_id":"1","upstream_id":"2391","short_description":"The classic aging pattern is defined as 2 decline on which subtest(s) of the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Arithmetic and Digit Span subtests', 'All subtests except Block Design', 'Similarities and Picture Completion subtests', 'Vocabulary subtest only', 'All Performance subtests', 'Block Design subtest only', 'All subtests except Vocabulary', 'Information and Comprehension subtests', 'All Verbal subtests']\",\"question\":\"The classic aging pattern is defined as 2 decline on which subtest(s) of the Wechsler Adult Intelligence Scale-Revised\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}