{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a61d2474-9a99-56fe-8143-456832cbd307","task_key":"default--test--4874","task_revision_id":"1","upstream_id":"4874","short_description":"The Younger Dryas was:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['a period of environmental change that caused humans to temporarily abandon much of northern Europe.', 'a warming period at the end of the Pleistocene.', 'a period of glacial expansion between 12,900 and 11,600 years ago.', 'a period of heavy meteorite impact causing a mass extinction event.', 'a period of significant volcanic activity that led to a global cooling.', 'a period of glacial retreat between 10,900 and 9,000 years ago.', 'a cooling period that occurred during the Jurassic period.', 'a period of rapid desertification in northern Africa.', 'a period of glacial retreat that marked the beginning of the Holocene epoch.', 'a period of glacial expansion during the Miocene epoch.']\",\"question\":\"The Younger Dryas was:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}