{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a9979c0-d145-571b-b777-629bb8da6776","task_key":"all--test--1a9979c0-d145-571b-b777-629bb8da6776","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Victories against outsiders granted legitimacy to leaders of states.', 'Victories against outsiders provided slaves whose labor sustained agricultural systems.', 'Victories against outsiders demonstrated the power of divinities over other gods.', 'Victories against outsiders allowed leaders of states to incorporate the elites of the subjugated into the conquering power.']\",\"question\":\"This question refers to the following information.\\nThis ruler of Syria made me spend many years as commander of his army,\\nEvery land to which I turned\\nI overcame.\\nI destroyed its green fields and its wells,\\nI captured its cattle, I took captive its inhabitants, I deprived them of their provisions,\\nand I slew [many] people…by my sword, my bow, my marchings, and my good devices.\\nThus my excellence was in his heart; he loved me and he knew my valor;\\n…he set me at the head of his sons, when he saw the success of my handiwork.\\nThere came a champion of Syria\\nto defy me in my tent;\\na bold man without equal, for he had vanquished all his rivals.\\nHe said, \\\"Let Sanehat fight with me.\\\"\\nHe thought to overcome me; he designed to take my cattle, thus being counseled by his tribe.\\nThe Tale of Sanehat, Egyptian poem written during the Middle Kingdom, ca.1800 B.C.E.\\nWhich of the following best describes the most common political significance of conflict between states to the rulers of early civilizations?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}