{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6e27003-0ce6-5c64-b41d-a17db25b7da1","task_key":"default--test--4754","task_revision_id":"1","upstream_id":"4754","short_description":"That modern Africans do not possess any Neandertal DNA indicates that:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Gene flow between Neandertals and anatomically modern humans must have occurred before Neandertals migrated out of Africa.', 'Gene flow between Neandertals and anatomically modern humans occurred after anatomically modern humans migrated out of Africa.', 'Neandertals who lived in Africa did not interbreed with anatomically modern humans, but must have followed them when they migrated out of Africa.', 'Interbreeding between anatomically modern humans and Neandertals never occurred.']\",\"question\":\"That modern Africans do not possess any Neandertal DNA indicates that:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}