{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b694e68-1396-5774-951a-3bc69149baf9","task_key":"default--test--5663","task_revision_id":"1","upstream_id":"5663","short_description":"Considering Wendt's three cultures of anarchy; which of the following defines…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['A culture characterized by friendship, where force and violence is eschewed in favour of cooperation in matters of security.', 'A culture in which actors are completely self-reliant and do not seek help from others.', 'A culture within which survival relies on military power, security dilemmas abound, and security is a zero-sum game.', 'A culture characterized by complete distrust, where actors only rely on their own military power for survival.', 'A culture where actors regard each other as threats but do not resort to violence.', 'A culture characterized by mutual understanding and inter-state cooperation without the need for military power.', 'A culture characterized by rivalry, which has dominated since the beginning of the modern system of states and in which actors regard each other as rivals but exercise some restraint in violence.', 'A culture where every state is for itself, and there is no cooperation between actors.', 'A culture characterized by constant warfare with no restraints on violence.', 'A culture that exists some of the time, but not all the time, in which no self-restraint exists and actors cannot rely on each other for help.']\",\"question\":\"Considering Wendt's three cultures of anarchy; which of the following defines the Lockean culture?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}