{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a3d902ec-f9f3-5550-bf89-8e569f68670c","task_key":"default--test--5075","task_revision_id":"1","upstream_id":"5075","short_description":"According to Krause and Williams, what agenda would most likely attract scholars…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['To question the referent object, to question the traditional conception of military security as paramount to the state, and to change the way in which security is studied.', 'To address security from a positivist perspective, challenging the dominance of traditional security paradigms, and shifting academic attention to the individual as a referent point.', 'To question the dominance of liberalism and realism in the international system, to apply critical knowledge to the security dilemma faced by states and to redress the power imbalances that lead to intra-state conflict.', 'To account for all possible referent objects and expand the definition of security so far as to allow for a broadening of the school to encompass any agent that threatens those referent objects.']\",\"question\":\"According to Krause and Williams, what agenda would most likely attract scholars to the 'broad church' of security studies?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}