{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23eb3f76-b030-5be2-bd07-46472ae93a5b","task_key":"all--test--23eb3f76-b030-5be2-bd07-46472ae93a5b","task_revision_id":"1","upstream_id":"","short_description":"The controversial theory that people from present-day Europe colonized North…","config":"all","split":"test","body":"{\"choices\":\"['supported by genetic evidence that establishes close connections between the oldest skeletons in the Americas and the European genome.', 'refuted by genetic evidence from the oldest skeletons in the Americas that establishes close connections between modern Native Americans and the native people of central Siberia.', 'proven by genetic evidence that establishes close connections between the oldest skeletons in the Americas and Norse explorers who arrived in northeastern Canada around A.D. 1000.', 'refuted by the complete absence of documents that record a migration of Europeans into the Americas.']\",\"question\":\"The controversial theory that people from present-day Europe colonized North America at the end of Pleistocene is:\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}