{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dbeef604-1382-59bb-b9f9-30e3974baeb1","task_key":"all--test--dbeef604-1382-59bb-b9f9-30e3974baeb1","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['His exile in Switzerland and other places led him to see how the industrialization of the West was needed in the Soviet Union.', 'Russia had a large percentage of its population working in factories before the revolution occurred, and he wanted to build on that.', 'Trotsky was able to lead the Bolshevik revolt in 1917 because he had control of the rail and telegraph stations.', 'The humiliating defeat of Russia during the First World War had led to his rise to power and he knew another defeat would spell his doom.']\",\"question\":\"This question refers to the following information.\\nThe following excerpt is from a speech by Vladimir I. Lenin.\\nThe independence of our country cannot be upheld unless we have an adequate industrial basis for defense. And such an industrial basis cannot be created if our industry is not more highly developed technically. That is why a fast rate of development of our industry is necessary and imperative. We cannot go on indefinitely, that is, for too long a period, basing the Soviet regime and socialist construction on two different foundations, the foundation of the most large-scale and united socialist industry and the foundation of the most scattered and backward, small commodity economy of the peasants. We must gradually, but systematically and persistently, place our agriculture on a new technical basis, the basis of large-scale production, and bring it up to the level of socialist industry. Either we accomplish this task—in which case the final victory of socialism in our country will be assured, or we turn away from it and do not accomplish it—in which case a return to capitalism may become inevitable.\\n—Vladimir I. Lenin Industrialization of the Country and the Right Deviation in the C.P.S.U., November 19,1928\\nWhich of the following events was most likely the greatest influence on Lenin in his desire to industrialize his new nation?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}