{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51c40b88-d94a-5120-8f77-233086343f81","task_key":"default--test--5148","task_revision_id":"1","upstream_id":"5148","short_description":"What differences emerged with regard to theorizing on both sides of the…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['From a US perspective, it is common to regard the main voice of security to be external. The question of what should or what should not count as security issues and how to conceptualize security.', 'The European school is based on a hegemonic form of knowledge. Knowledge is conceived as the understanding of causal mechanisms in relation to the material on which they act.', 'The different forms of knowledge relate to a conflicting conception of the relationship to policy: the problem-solving tradition in US political science vs. the critical studies in Europe.', 'Theorizing has parallel theoretical forms of knowledge based on empirically substantiated cause-and-effect relationships.']\",\"question\":\"What differences emerged with regard to theorizing on both sides of the Atlantic?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}