{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54bb6168-5b36-5ac4-8ae1-1fadc5af2a2a","task_key":"default--test--5504","task_revision_id":"1","upstream_id":"5504","short_description":"Some governmental regimes are reluctant to fully join the liberalist camp; which…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The dominance of Western ideologies can be regarded as contemporary imperialism.', 'Liberalism advocates the enforcement of certain values that are not shared by all political regimes.', 'The fear that cooperation may be recurring.', 'Adherence to liberalism does not guarantee the favoured treatment of state regimes over its citizens; other states may threaten regime security in their efforts to secure societal security.']\",\"question\":\"Some governmental regimes are reluctant to fully join the liberalist camp; which of the following is not a valid reason for this reluctance?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}