{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a5e674e-b4e8-5a25-b78d-ace68becbe8b","task_key":"default--test--5592","task_revision_id":"1","upstream_id":"5592","short_description":"Which of the following ways is NOT a defining feature that distinguishes…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Postcolonialism primarily focuses on the role of economic factors in security studies.', 'Postcolonialism is not a predominantly or exclusively European form of knowledge.', \\\"Postcolonialism doesn't consider the influence of colonial history on current security issues.\\\", 'Postcolonialism does not consider the impact of global inequality on security.', 'Postcolonialism uniquely considers the nation or community group as a valid referent object in security studies.', 'Postcolonialism employs a distinctive focus on the role of violence in world politics.', 'Postcolonialism concentrates on bilateral rather than multilateral security issues.', 'Postcolonialism disregards the influence of cultural factors in security studies.', \\\"Postcolonialism takes account not only of major war, but also of so-called 'small wars' and asymmetric conflicts.\\\"]\",\"question\":\"Which of the following ways is NOT a defining feature that distinguishes postcolonialism from other perspectives on security?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}