{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58feae1d-2ba8-5120-9609-12d75e9955d5","task_key":"all--test--58feae1d-2ba8-5120-9609-12d75e9955d5","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['it pledged Germany to join the Triple Alliance and support Austria-Hungary against the Triple Entente', 'it was understood to give Austria an unlimited scope of response to the assassination of Franz Ferdinand, to Serbia, and Pan-Slavism within the Austro-Hungarian Empire', \\\"it pledged Germany's unlimited support to Franz Joseph in his efforts to succeed the recently assassinated Franz Ferdinand\\\", 'it offered nothing in terms of real support to Franz Joseph and Austria-Hungary']\",\"question\":\"This question refers to the following information.\\nConfidential—For Your Excellency's personal information and guidance\\nThe Austro-Hungarian Ambassador yesterday delivered to the [German] Emperor [Wilhelm II] a confidential personal letter from the Emperor Francis Joseph [of Austria-Hungary], which depicts the present situation from the Austro-Hungarian point of view, and describes the measures which Vienna has in view. A copy is now being forwarded to Your Excellency. …\\nHis Majesty desires to say that he is not blind to the danger which threatens Austria-Hungary and thus the Triple Alliance as a result of the Russian and Serbian Pan-Slavic agitation. … His Majesty will, furthermore, make an effort at Bucharest, according to the wishes of the Emperor Franz Joseph, to influence King Carol to the fulfilment of the duties of his alliance, to the renunciation of Serbia, and to the suppression of the Rumanian agitations directed against Austria-Hungary.\\nFinally, as far as concerns Serbia, His Majesty, of course, cannot interfere in the dispute now going on between Austria-Hungary and that country, as it is a matter not within his competence. The Emperor Franz Joseph may, however, rest assured that His Majesty will faithfully stand by Austria-Hungary, as is required by the obligations of his alliance and of his ancient friendship.\\nTheobald von Bethmann-Hollweg (chancellor of Germany), telegram to the German ambassador at Vienna, July 6, 1914\\nFrom the passage, one may infer that Bethmann-Hollweg's telegram is often referred to as Germany's \\\"blank check\\\" because\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}