{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"881d831c-e8f1-555b-aa71-7249d2c7e3a1","task_key":"all--test--881d831c-e8f1-555b-aa71-7249d2c7e3a1","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Responses to failed unification attempts in Africa', 'A plea for the United Nations to intervene in Africa', 'A proposal to increase the standing of Africa in the modern world', 'Ardent opposition to the influence of capitalism in Africa']\",\"question\":\"This question refers to the following information.\\n\\\"The greatest contribution that Africa can make to the peace of the world is to avoid all the dangers inherent in disunity, by creating a political union which will also by its success, stand as an example to a divided world. A Union of African states will project more effectively the African personality. It will command respect from a world that has regard only for size and influence. The scant attention paid to African opposition to the French atomic tests in the Sahara, and the ignominious spectacle of the U.N. in the Congo quibbling about constitutional niceties while the Republic was tottering into anarchy, are evidence of the callous disregard of African Independence by the Great Powers.\\\"\\nKwame Nkrumah, Ghanian leader, 1961\\nThe passage is most clearly an example of which of the following?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}