{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26e3470d-9f0a-5e7e-89d5-9d7965ff7aa2","task_key":"all--test--26e3470d-9f0a-5e7e-89d5-9d7965ff7aa2","task_revision_id":"1","upstream_id":"","short_description":"The process of invasion and succession is a process involving migration and is…","config":"all","split":"test","body":"{\"choices\":\"['new residents tear down and rebuild housing units.', 'new residents live peacefully with old residents.', 'new residents assimilate with old residents completely.', 'new residents move into areas occupied by older resident groups.']\",\"question\":\"The process of invasion and succession is a process involving migration and is best described as when\",\"subject\":\"high_school_geography\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}