{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4704cf4-69bd-55cb-9880-51e54e9ea2d0","task_key":"all--test--a4704cf4-69bd-55cb-9880-51e54e9ea2d0","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"[\\\"resulted from Germany's defeat in World War II\\\", \\\"was unaffected by Germany's defeat in World War II\\\", \\\"was proven correct by Germany's defeat in World War II\\\", \\\"was discredited by Germany's defeat in World War II\\\"]\",\"question\":\"This question refers to the following information.\\nFor a long time, educated Germans answered it in the positive, initially by laying claim to a special German mission, then, after the collapse of 1945, by criticizing Germany's deviation from the West. Today, the negative view is predominant. Germany did not, according to the now prevailing opinion, differ from the great European nations to an extent that would justify speaking of a \\\"unique German path.\\\" And, in any case, no country on earth ever took what can be described as the \\\"normal path.\\\"\\nHeinrich August Winkler, Germany: The Long Road West, Volume 1, 2006\\nFrom this passage, one may infer that the belief that Germany had a unique mission in history\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}