{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0add177f-c5f4-56f8-8562-987665832989","task_key":"all--test--0add177f-c5f4-56f8-8562-987665832989","task_revision_id":"1","upstream_id":"","short_description":"What was concluded in a recent study comparing the genomes of modern people with…","config":"all","split":"test","body":"{\"choices\":\"['Native Americans in North America and South America genetically diverged from each other and their ancestors in the far north by about 30,000 years ago.', 'All Native Americans, both ancient and modern, are derived from a single migration of Austral-Melanesians into the Americas by about 13,000 years ago.', 'All Native Americans, both ancient and modern, are derived from a single migration of the Inuit, who arrived in the Americas by about 13,000 years ago.', 'The separation between native Siberians and the earliest Native Americans occurred sometime between 20,000 and 23,000 years ago.']\",\"question\":\"What was concluded in a recent study comparing the genomes of modern people with ancient skeletal remains in North and South America?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}