{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac98d786-3c5d-5970-9e39-64345c2e3a21","task_key":"all--test--ac98d786-3c5d-5970-9e39-64345c2e3a21","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['a restructuring of the socialist society of the Soviet Union was inevitable', 'a restructuring of the socialist society of the Soviet Union would exacerbate the current situation', 'a restructuring of the socialist society of the Soviet Union was necessary in order to avoid a crisis', 'a restructuring of socialist society was necessary in order to avoid the collapse of the Soviet Union']\",\"question\":\"This question refers to the following information.\\nPerestroika [Restructuring] is an urgent necessity arising from the profound processes of development in our socialist society. This society is ripe for change. It has long been yearning for it. Any delay in beginning perestroika could have led to an exacerbated internal situation in the near future, which, to put it bluntly, would have been fraught with serious social, economic, and political crises.\\nMikhail Gorbachev, Perestroika: New Thinking for Our Country and the World, 1987\\nFrom the passage, one may infer that Gorbachev argued that\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}