{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c47ab6fd-4ba7-51e8-9daa-c3dfe0d460da","task_key":"all--test--c47ab6fd-4ba7-51e8-9daa-c3dfe0d460da","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Capitalism', 'Mercantilism', 'Fascism', 'Marxism']\",\"question\":\"This question refers to the following information.\\nSource 1:\\n\\\"In the days of a great struggle against a foreign enemy who has been endeavoring for three years to enslave our country, it pleased God to send Russia a further painful trial. Internal troubles threatened to have a fatal effect on the further progress of this obstinate war. The destinies of Russia, the honor of her heroic Army, the happiness of the people, and the whole future of our beloved country demand that the war should be conducted at all costs to a victorious end.\\nThe cruel enemy is making his last efforts and the moment is near when our valiant Army, in concert with our glorious Allies, will finally overthrow the enemy. In these decisive days in the life of Russia we have thought that we owed to our people the close union and organization of all its forces for the realization of a rapid victory; for which reason, in agreement with the Imperial Duma, we have recognized that it is for the good of the country that we should abdicate the Crown of the Russian State and lay down the Supreme Power.\\\"\\nTsar Nicholas II, Abdication, March 15, 1917\\nSource 2:\\n\\\"History will not forgive revolutionaries for procrastinating when they could be victorious today (and they certainly will be victorious today), while they risk losing much tomorrow, in fact, the risk losing everything.\\nIf we seize power today, we seize it not in opposition to the Soviets but on their behalf. The seizure of power is the business of the uprising; its political purpose will become clear after the seizure….\\nIt would be an infinite crime on the part of the revolutionaries were they to let the chance slip, knowing that the salvation of the revolution, the offer of peace, the salvation of Petrograd, salvation from famine, the transfer of the land to the peasants depend upon them.\\nThe government is tottering. It must be given the death-blow at all costs.\\\"\\nVladimir Illyich Lenin, Call to Power, October 24, 1917\\nWhat was the principal philosophical underpinning of Lenin's call to power in Source 2?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}