{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d50d12a5-3147-546d-b8ad-4c6e6e0ddef7","task_key":"all--test--d50d12a5-3147-546d-b8ad-4c6e6e0ddef7","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['the failure to bring about a change in the ruling class', 'the demonstration of the power of ideas', 'the discrediting of both conservative and liberal political ideology', 'the creation of the idea of a modern police force']\",\"question\":\"This question refers to the following information.\\nRead the the following quotation to answer questions.\\n1848 was the decisive year of German, and so of European, history: it recapitulated Germany's past and inspired Germany's future. … Never has there been a revolution so inspired by a limitless faith in the power of ideas; never has a revolution so discredited the power of ideas in its result. The success of the revolution discredited conservative ideas; the failure of the revolution discredited liberal ideas. After it, nothing remained but the idea of Force, and this idea stood at the helm of German history from then on. For the first time since 1521, the German people stepped on to the centre of the German stage only to miss their cue once more. German history reached its turning-point and failed to turn. This was the fateful essence of 1848.\\nA. J. P. Taylor, The Course of German History, 1945\\nFrom the quotation, one may infer that Taylor argues that the most important effect of the political revolutions of 1848 was\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}