{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce832f78-54e3-5175-a514-11db0408a8ca","task_key":"all--test--ce832f78-54e3-5175-a514-11db0408a8ca","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Collectivization', 'Persecution of the better-off peasants', 'The Berlin blockade', 'The five-year plans']\",\"question\":\"This question refers to the following information.\\nThe following excerpt is from a speech by Vladimir I. Lenin.\\nThe independence of our country cannot be upheld unless we have an adequate industrial basis for defense. And such an industrial basis cannot be created if our industry is not more highly developed technically. That is why a fast rate of development of our industry is necessary and imperative. We cannot go on indefinitely, that is, for too long a period, basing the Soviet regime and socialist construction on two different foundations, the foundation of the most large-scale and united socialist industry and the foundation of the most scattered and backward, small commodity economy of the peasants. We must gradually, but systematically and persistently, place our agriculture on a new technical basis, the basis of large-scale production, and bring it up to the level of socialist industry. Either we accomplish this task—in which case the final victory of socialism in our country will be assured, or we turn away from it and do not accomplish it—in which case a return to capitalism may become inevitable.\\n—Vladimir I. Lenin Industrialization of the Country and the Right Deviation in the C.P.S.U., November 19,1928\\nWhich of the following policies of Joseph Stalin was NOT used to correct the problem cited by Lenin above?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}