{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2369c136-2967-508f-8b46-befe222d6051","task_key":"all--test--2369c136-2967-508f-8b46-befe222d6051","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The rise of Sikhism as a syncretic faith', 'The arrival of Islam as a new religion borne by external invaders', 'The eradication of traditional Hinduism by armed force', 'The official encouragement of Buddhist edicts']\",\"question\":\"This question refers to the following information.\\nKnow the self to be sitting in the chariot, the body to be the chariot, the intellect the charioteer, and the mind the reins. He who has understanding, who is mindful and always pure, indeed reaches that place from whence he is not born again.\\nfrom the Upanishads, India, circa 600s–400s B.C.E.\\nWhich of the following changes to Indian religious practice took place under the Mauryan emperor Ashoka?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}