{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72760929-cc2f-5938-b8cf-a828707df506","task_key":"default--test--5294","task_revision_id":"1","upstream_id":"5294","short_description":"In the context of military security, which of the following is not an obvious…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['A more critical approach to military security studies will allow the inclusion or adoption of non-realist paradigms, such as social constructivism.', 'Critical analysis of military security will draw attention back to traditional conceptions of the state-centric paradigm which have been neglected in recent years, with states favouring human security and other non-traditional approaches instead.', 'By implementing a critical analysis we can recognise that traditional conceptions of military security are no longer always salient and may need to be reconsidered.', 'A critical approach would recognise the changing ontology of security in the real world. Such changes are evident in the new ways in which war has been conducted in the 20th and 21st century.']\",\"question\":\"In the context of military security, which of the following is not an obvious benefit of adopting a 'critical' theoretical approach?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}