{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97aedc84-2311-5e4b-b660-f8be125f8f57","task_key":"all--test--97aedc84-2311-5e4b-b660-f8be125f8f57","task_revision_id":"1","upstream_id":"","short_description":"There is general consensus among paleoanthropologists that anatomically modern…","config":"all","split":"test","body":"{\"choices\":\"['migrated out of Africa into western Europe between 150,000 and 200,000 years ago and then spread into Asia, Australia, and the Americas.', 'evolved in Asia between 100,000 and 50,000 years ago, and then migrated into Africa, Europe, Australia, and the Americas.', 'evolved independently and simultaneously in Africa, Europe, and Asia and then migrated into Australia and the Americas.', 'migrated out of Africa into Southwest Asia between 150,000 and 100,000 years ago and then spread into Europe, Asia, Australia, and the Americas.']\",\"question\":\"There is general consensus among paleoanthropologists that anatomically modern humans:\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}