{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a17281d0-e0f0-5562-af18-462193734643","task_key":"default--test--5039","task_revision_id":"1","upstream_id":"5039","short_description":"There is general consensus among paleoanthropologists that anatomically modern…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['migrated out of Africa into Southwest Asia between 150,000 and 100,000 years ago and then spread into Europe, Asia, Australia, and the Americas.', 'evolved in Asia between 100,000 and 50,000 years ago, and then migrated into Africa, Europe, Australia, and the Americas.', 'migrated out of Asia into Africa between 200,000 and 150,000 years ago and then spread into Europe, Australia, and the Americas.', 'evolved in Africa between 100,000 and 50,000 years ago, and then migrated into Europe, Asia, Australia, and the Americas.', 'migrated out of Africa into Asia between 100,000 and 50,000 years ago and then spread into Europe, Australia, and the Americas.', 'migrated out of Africa into western Europe between 150,000 and 200,000 years ago and then spread into Asia, Australia, and the Americas.', 'evolved independently in Africa, Europe, and Asia between 200,000 and 150,000 years ago and then migrated into Australia and the Americas.', 'evolved in Europe between 150,000 and 100,000 years ago and then migrated into Africa, Asia, Australia, and the Americas.', 'migrated out of Africa into North America between 150,000 and 100,000 years ago and then spread into Europe, Asia, and Australia.', 'evolved independently and simultaneously in Africa, Europe, and Asia and then migrated into Australia and the Americas.']\",\"question\":\"There is general consensus among paleoanthropologists that anatomically modern humans:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}