{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47d75d2e-6265-54ce-96ce-51de11ec9177","task_key":"default--test--4988","task_revision_id":"1","upstream_id":"4988","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['an attempt to bring civilization to uncivilized peoples', 'a mission to educate and enlighten', 'a task that is doomed to fail', 'a glorious, religious responsibility', 'a profitable endeavor for the empire', 'a difficult but necessary task', 'a necessary evil for the good of the empire', 'a ruthless exercise of power', 'a hopeless task with no purpose', 'a burden that should be willingly undertaken']\",\"question\":\"This question refers to the following information.\\nRead the the following poem to answer questions.\\nTake up the White Man's burden—\\nSend forth the best ye breed—\\nGo bind your sons to exile\\nTo serve your captives' need;\\nTo wait in heavy harness,\\nOn fluttered folk and wild—\\nYour new-caught, sullen peoples,\\nHalf-devil and half-child.\\n...\\nTake up the White Man's burden—\\nThe savage wars of peace—\\nFill full the mouth of Famine\\nAnd bid the sickness cease;\\nAnd when your goal is nearest\\nThe end for others sought…\\nWatch sloth and heathen Folly\\nBring all your hopes to naught.\\nRudyard Kipling, The White Man's Burden, 1899\\nBased on this poem, Kipling's conception of the British imperial mission could best be summed up as\",\"src\":\"ori_mmlu-high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}