{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c3684e3-cee8-5a30-8ee2-53527702acb6","task_key":"all--test--5c3684e3-cee8-5a30-8ee2-53527702acb6","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['For there is nothing better for a Kshatriya than a righteous battle.', 'The four divisions of human society are created by me [Krishna].', 'The person traditionally credited with composing the Mahabarata was born to a fisherwoman.', 'The Hindu God Shiva is considered to be simultaneously the creator, the preserver, and the destroyer.']\",\"question\":\"This question refers to the following information.\\n\\\"Purusa-Sukta\\\"\\nPurusa is the lord of the immortals, who grow by means of [ritual] food. When the gods performed a sacrifice with the offering Purusa, spring was its clarified butter, summer the kindling, autumn the oblation.\\nIt was Purusa, born in the beginning, which they sprinkled on the sacred grass as a sacrifice. It made the beasts of the air, the forest and the village. From that sacrifice completely offered, the mantras [Rig Veda] and the songs [Samaveda] were born. The sacrificial formulae [Yajurveda] were born from it. From it the horses were born and all that have cutting teeth in both jaws. The cows were born from it, also. From it were born goats and sheep.\\nWhen they divided Purusa, how many ways did they apportion him? What was his mouth? What were his arms? What were his thighs, his feet declared to be? His mouth was the Brahman, his arms were the Rajanaya [Ksatriya], his thighs the Vaisya; from his feet the Sudra was born. Thus, they fashioned the worlds. The gods sacrificed with the sacrifice to the sacrifice. These were the first rites.\\n—Hymns excerpted and adapted from the Rig Veda, oldest surviving literary work, India, 1500–1000 BCE\\nWhich of the following would best contradict the argument that the caste system imposed rigid economic and social roles on Indian society in the period 600 BCE to 600 CE?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}