{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83ecad2e-fe5f-5a1a-bdae-7c1e8e2884e9","task_key":"all--test--83ecad2e-fe5f-5a1a-bdae-7c1e8e2884e9","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Ancient Egyptian society held compassion up as the highest virtue.', \\\"Ancient Egyptians believed that moral behavior affected one's fate after death.\\\", 'Ancient Egyptian clergy promoted a strict \\\"eye for an eye\\\" code of ethics.', 'Ancient Egyptian worshippers committed to an elaborate set of rituals and offerings.']\",\"question\":\"This question refers to the following information.\\nSeizing me, he led me down to the House of Darkness…\\nTo the house where those who enter do not come out,\\nAlong the road of no return,\\nTo the house where those who dwell, do without light,\\nWhere dirt is their drink, their food is of clay…\\nWhere light cannot be seen. They dwell in the dark,\\nAnd upon the door and bolt, there lies dust.\\nfrom Tablet VII, The Epic of Gilgamesh, Sumeria, ca. 2100 B.C.E.\\nTruth is good and its worth is lasting.\\nIt has not been disturbed since the day of its creator,\\nWhereas he who transgresses its ordinances is punished.\\nIt lies as a path in front of even him who knows nothing.\\nWrongdoing has never yet brought its venture to port…\\nThose whom the gods guide cannot lose themselves.\\nThose to whom they forbid passage will not be able to cross the river of life.\\nfrom The Maxims of Ptahhotep, Egypt, ca. 2400 B.C.E.\\nThe tone of the second text best reflects which of the following hypotheses about the influence of religious beliefs on ancient societies?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}