{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"887a290d-da3d-557b-b3dc-3276c58a42a7","task_key":"all--test--887a290d-da3d-557b-b3dc-3276c58a42a7","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['provoking fear to drive their readers to action', 'attacking the extreme communists who threatened to sink the entire movement', 'demonstrating the understanding of the others in point of fact', 'explaining the reasons for debunking opponents of communism']\",\"question\":\"This question refers to the following information.\\n\\\"A specter is haunting Europe-the specter of Communism. All the powers of old Europe have entered into a holy alliance to exorcise this specter; Pope and Czar, Metternich and Guizot, French radicals and German police spies.\\nTwo things result from this fact.\\nI. Communism is already acknowledged by all European powers to be in itself a power.\\nII. It is high time that Communists should openly, in the face of the whole world, publish their views, their aims, their tendencies, and meet this nursery tale of the Specter of Communism with a Manifesto of the party itself.\\\"\\nKarl Marx and Friedrich Engels, The Communist Manifesto, 1848\\nIn this passage, Marx and Engels seem to be chiefly preoccupied with\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}