{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa5cce5a-5173-5774-8a1d-11bdf554062e","task_key":"all--test--aa5cce5a-5173-5774-8a1d-11bdf554062e","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Counterrevolutionary opponents', 'Foreign invaders', 'Communist agitators', 'Transnational corporations']\",\"question\":\"This question refers to the following information.\\nThis great purity of the French Revolution is precisely what causes both our strength and our weakness. Our strength, because it gives to us rights of the public interest over private interests; our weakness, because it rallies all vicious men against us. We must smother the internal and external enemies of the Republic or perish with it; now in this situation, the first maxim of your policy ought to be to lead the people by reason and the people's enemies by terror. Terror is nothing other than justice, prompt, severe, inflexible; it is therefore an emanation of virtue; it is not so much a special principle as it is a consequence of the general principle of democracy applied to our country's most urgent needs.\\nFrench revolutionary leader Maximilien Robespierre, 1794\\nWith which of the following threats does the speaker in the passage above seem most concerned?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}