{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"171a3375-89e4-57f8-a58d-4b21fb32c404","task_key":"all--test--171a3375-89e4-57f8-a58d-4b21fb32c404","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Ancient leaders were valued for their mastery of complicated legal codes.', 'Ancient leaders were expected to be experts of animal husbandry.', 'Ancient leaders were given divine origins to bolster their legitimacy.', 'Ancient leaders were not judged on their appearances.']\",\"question\":\"This question refers to the following information.\\n\\\"When the gods fashioned Gilgamesh*, to him they gave a perfect form. The glorious sun Shamash bestowed upon him glory; Adad the terrible god of storms bestowed upon him courage. The great gods perfected his magnificence beyond all others, terrible like the great wild bull. Two thirds god they made him; one third man they made him.\\nOf Uruk, its great rampart he built, and the wall of the sacred Eanna temple, the holy sanctuary. Behold the outer walls which gleam with the brilliance of copper; see the inner wall which none might rival. Touch the threshold stone—it is from ancient days. Goest thou into the Eanna temple, yea, the dwelling place of Ishtar, the like of which no subsequent king or living man might equal. Ascend and walk about on the wall of Uruk, inspect the corner-stone, and examine its brick-work, whether its wall is not made of burned brick, and its foundation laid by the Seven Sages. One third for city, one third for garden, one third for field, and a precinct for the temple of Ishtar. These parts and the precinct comprise Uruk.\\\"\\nThe Epic of Gilgamesh, ca. 2100 B.C.E.\\n*Gilgamesh was a king of Uruk.\\nThe description of Gilgamesh in the first paragraph of the passage most directly supports which of the following statements about ancient leaders?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}