{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d126db02-ab38-5205-b512-2c7b4498e2e3","task_key":"all--test--d126db02-ab38-5205-b512-2c7b4498e2e3","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['the problems that required perestroika were the fault of capitalist enemies of socialism', 'the problems that required perestroika were internal to the development of socialist society', 'a socialist society could not work', 'a socialist society could not coexist with capitalism']\",\"question\":\"This question refers to the following information.\\nPerestroika [Restructuring] is an urgent necessity arising from the profound processes of development in our socialist society. This society is ripe for change. It has long been yearning for it. Any delay in beginning perestroika could have led to an exacerbated internal situation in the near future, which, to put it bluntly, would have been fraught with serious social, economic, and political crises.\\nMikhail Gorbachev, Perestroika: New Thinking for Our Country and the World, 1987\\nFrom the passage, one may infer that Gorbachev believed that\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}