{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4886670-4f09-5162-9833-a21bec95a859","task_key":"default--test--4674","task_revision_id":"1","upstream_id":"4674","short_description":"Recent discoveries of Homo naledi fossils in the Dinaledi Chamber of the Rising…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the Homo naledi fossils were found in the same layer as modern human fossils, suggesting they lived at the same time.', 'the Homo naledi fossils have traits that do not fit into the current understanding of hominid evolution.', 'paleoanthropologists did not make the discovery and it was found in an inaccessible cave in South Africa.', 'the fossils were found in a region previously thought to be inhabited by a different species of Homo.', 'the Homo naledi fossils show traits of both Homo and Australopithecus, suggesting a potential hybrid species.', 'the methods used to extract the fossils from the cave were controversial and potentially damaging to the specimens.', 'there is, as yet, no firm date that can be applied to the specimens.', 'the Homo naledi fossils have been shown to pre-date the earliest known hominid fossils by millions of years.', 'there are already two well-known species of Homo and it is unlikely there were more than two species at one time.', 'the researchers who discovered the fossils have been accused of falsifying their findings.']\",\"question\":\"Recent discoveries of Homo naledi fossils in the Dinaledi Chamber of the Rising Star cave system in South Africa suggest a mixture of post-cranial morphologies or mosaic of hominid traits. One of the reasons this new species is controversial is that:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}