{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1012ba0d-2db3-5c08-bd44-b3fcd6af44ae","task_key":"default--test--5742","task_revision_id":"1","upstream_id":"5742","short_description":"What is the relationship between war, military security, and security studies?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['War is a strategic tool used by states to assert dominance and increase their security.', 'War is best regarded as a rational instrument of political policy; it is a continuation of politics by other means.', 'War is best regarded as a social aberration, and is the result of catastrophic failure in social cohesion.', 'War is a consequence of the failure of diplomatic negotiations and is the last resort for conflict resolution.', 'War is a result of the failure of security studies to produce effective peacekeeping strategies.', 'War is an anarchic state in which state or other actors must draw upon exceptional (military) means in order to ensure the protection of their interests.', 'War is a state of mass conflict which is inflicted upon state actors as a result of their failure to effectively control the behaviour of their population.', 'War is a necessary means to maintain international balance and security.', 'War is a state of conflict that arises from the absence of effective international security structures.']\",\"question\":\"What is the relationship between war, military security, and security studies?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}