{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3a54d77-1d03-5006-8723-887d208dfbe0","task_key":"all--test--f3a54d77-1d03-5006-8723-887d208dfbe0","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Demonstrate the possibility of African unity', 'Give an example of how African nations are neglected by the world powers', 'Present an illustration of how the development of a constitution brings order to nations', 'Blame European powers for the difficulties experienced in Africa']\",\"question\":\"This question refers to the following information.\\n\\\"The greatest contribution that Africa can make to the peace of the world is to avoid all the dangers inherent in disunity, by creating a political union which will also by its success, stand as an example to a divided world. A Union of African states will project more effectively the African personality. It will command respect from a world that has regard only for size and influence. The scant attention paid to African opposition to the French atomic tests in the Sahara, and the ignominious spectacle of the U.N. in the Congo quibbling about constitutional niceties while the Republic was tottering into anarchy, are evidence of the callous disregard of African Independence by the Great Powers.\\\"\\nKwame Nkrumah, Ghanian leader, 1961\\nThe inclusion of the Congo in Nkrumah's argument is intended to do which of the following?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}