{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45d8360b-3470-5679-9898-33a638cc5a73","task_key":"default--test--5197","task_revision_id":"1","upstream_id":"5197","short_description":"What was the significance of the Truman Doctrine?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"It outlined Truman's plan for economic recovery after World War II\\\", 'It led to the establishment of the United Nations', 'It indicated that the US would now view all local revolts through a geopolitical lens', 'It was a declaration of war against the Soviet Union', 'It indicated the special place of Greece and Turkey in American interests', \\\"It signaled the US's withdrawal from international affairs\\\", 'It marked the beginning of the Space Race', 'It marked the end of the Cold War', \\\"It was Truman's first statement on European affairs\\\", 'It indicated US reluctance to get involved in incidents outside of its immediate sphere of influence']\",\"question\":\"What was the significance of the Truman Doctrine?\",\"src\":\"ori_mmlu-us_foreign_policy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}