{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"275dfb33-89b7-5594-8599-3ed169f2b198","task_key":"default--test--2390","task_revision_id":"1","upstream_id":"2390","short_description":"Numerous factor-analytic studies have been performed on tests of motor function…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['most motor functions are determined by age and gender', 'motor functions are independent and do not correlate', 'motor functions correlate moderately and form 3 to 4 factors', 'most motor functions correlate highly and form a single factor', 'most motor functions correlate highly and form 2 single factor', 'motor functions are best represented by physical fitness', 'two factors, speed and strength, account for most of the common variance', 'most motor functions are highly specific', 'three factors, speed, strength, and coordination, account for most of the common variance', 'most motor functions are well represented by intelligence']\",\"question\":\"Numerous factor-analytic studies have been performed on tests of motor function (e.g., Purdue Pegboard). In general, these studies have revealed that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}