{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bd2e46f-0679-5b71-a78b-28a9029356d3","task_key":"all--test--9bd2e46f-0679-5b71-a78b-28a9029356d3","task_revision_id":"1","upstream_id":"","short_description":"Many countries that became independent after World War II found that the legacy…","config":"all","split":"test","body":"{\"choices\":\"['artificial political boundaries that covered diverse ethnic groups with little in common', 'inadequate access to higher education and training for nationals under the colonial administration', 'control of domestic commerce by foreign minority groups which had been brought in by the colonial power', 'all of the above']\",\"question\":\"Many countries that became independent after World War II found that the legacy of colonialism hindered their economic development. The list of hindrances included\",\"subject\":\"global_facts\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}