{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1cdc231-0454-577b-9f35-b4bf26a1bcbf","task_key":"all--test--b1cdc231-0454-577b-9f35-b4bf26a1bcbf","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The ideals of the Russian Revolution were betrayed by the Leninist and Stalinist regimes.', 'Lenin and Stalin vastly improved Russian military preparedness.', 'Communism has been made to work more efficiently in Russia than in China.', 'Despite abuses on the part of Lenin and Stalin, the Soviet regime managed significant achievements.']\",\"question\":\"This question refers to the following information.\\nLiteracy rates among the Russian-speaking population of late Imperial Russia and the Soviet Union, 1897–1955\\nLiteracy Rate\\n1897  24%\\n1917  45%\\n1926  56%\\n1937  75%\\n1939  81.10%\\n1955  99.90%\\nSource: Census data and Soviet Ministry of Education\\nA historian sympathetic to Soviet communism would most likely use the chart as evidence to support which of the following propositions?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}