{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f118c074-00bc-5e17-8e1c-29938fd6e5a6","task_key":"all--test--f118c074-00bc-5e17-8e1c-29938fd6e5a6","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['retained the concept of karma.', 'proposed a heavenly existence after death.', 'rejected the validity of all gods.', 'mandated unquestioning obedience to the priestly class.']\",\"question\":\"This question refers to the following information.\\nKnow the self to be sitting in the chariot, the body to be the chariot, the intellect the charioteer, and the mind the reins. He who has understanding, who is mindful and always pure, indeed reaches that place from whence he is not born again.\\nfrom the Upanishads, India, circa 600s–400s B.C.E.\\nCompared to the message communicated by the excerpt, Buddhist doctrine regarding reincarnation most resembled Hinduism in that it\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}