{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d048ed2-8a87-557c-90fc-4450a656e3e3","task_key":"default--test--5076","task_revision_id":"1","upstream_id":"5076","short_description":"What external strategies do elites employ?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Elites may also use economic strategies, such as the manipulation of currency or the imposition of sanctions, to maintain their power and control.', 'An increasingly prevalent strategy among weak states has been the employment of private military or security companies and foreign mercenaries as force multipliers to secure the weak state from internal threats.', 'Weak states seek to avoid alliances with powerful states because alliances and assistance frequently are conditional on democratization and risk penalties if they are not met.', 'An increasingly popular means of bolstering regime security has been the expulsion of foreign workers and non-ethnic nationals and restricting borders to refugees and migrants.']\",\"question\":\"What external strategies do elites employ?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}