{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3523650-ac0d-574d-8869-2071ca5da695","task_key":"all--test--c3523650-ac0d-574d-8869-2071ca5da695","task_revision_id":"1","upstream_id":"","short_description":"What does the lid of the sarcophagus of Pacal the Great, ruler of Palenque,…","config":"all","split":"test","body":"{\"choices\":\"['the slain enemies of the king, with a series of hieroglyphs representing all of the great cities and civilizations of the world', 'the dead king suspended between the Maya earth monster and the Quetzal bird which symbolizes the Maya heaven', 'the Aztec king ascending to heaven on a chariot made of gold', 'an ancient alien looking through a telescope and piloting a spaceship']\",\"question\":\"What does the lid of the sarcophagus of Pacal the Great, ruler of Palenque, depict?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}