{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7802527-cdfd-5c78-a112-f5db4243b3d0","task_key":"default--test--1757","task_revision_id":"1","upstream_id":"1757","short_description":"Which of the following factors contributed significantly to the revival of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The end of the Cold War', 'The Bolshevik Revolution', 'The international recognition of human rights after World War II', 'The invention of the Internet', 'The rise of Fascism', 'The rise of Communism in China', 'Depression of the 1930', 'The advent of the Industrial Revolution']\",\"question\":\"Which of the following factors contributed significantly to the revival of natural law in the 20th century?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}