{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd19e38c-eebf-535c-96fc-d66a7338c627","task_key":"all--test--cd19e38c-eebf-535c-96fc-d66a7338c627","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['a difficult but necessary task', 'a ruthless exercise of power', 'a glorious, religious responsibility', 'a hopeless task with no purpose']\",\"question\":\"This question refers to the following information.\\nRead the the following poem to answer questions.\\nTake up the White Man's burden—\\nSend forth the best ye breed—\\nGo bind your sons to exile\\nTo serve your captives' need;\\nTo wait in heavy harness,\\nOn fluttered folk and wild—\\nYour new-caught, sullen peoples,\\nHalf-devil and half-child.\\n...\\nTake up the White Man's burden—\\nThe savage wars of peace—\\nFill full the mouth of Famine\\nAnd bid the sickness cease;\\nAnd when your goal is nearest\\nThe end for others sought…\\nWatch sloth and heathen Folly\\nBring all your hopes to naught.\\nRudyard Kipling, The White Man's Burden, 1899\\nBased on this poem, Kipling's conception of the British imperial mission could best be summed up as\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}