{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04e9c3eb-38c3-5baf-b776-aa85b3dffe17","task_key":"default--test--1380","task_revision_id":"1","upstream_id":"1380","short_description":"What is the meaning of collective security?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The establishment of a mutual defense pact among several nations', 'The right of a nation to defend itself without outside intervention', 'The authorisation of armed force by the UN Security Council', 'The authorisation of humanitarian missions by the UN Security Council', \\\"The right of one's allies to defend the victim State\\\", 'The authorisation of sanctions by the UN Security Council', 'The right to self-defence by more than one nation acting in concert', 'The right of a nation to invade another for its own security', 'The authorisation of peacekeeping missions by the UN General Assembly', 'The creation of a global police force by the UN']\",\"question\":\"What is the meaning of collective security?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}