{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec950c10-6ed5-5637-9099-b434164e0fa4","task_key":"default--test--5400","task_revision_id":"1","upstream_id":"5400","short_description":"Which of the following was a significant reason for the electoral success of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['A strong anti-war stance during World War II', 'A focus on mobilizing young voters', 'A commitment to ending racial segregation', 'A major shift of African American voters from the Republican to the Democratic Party', 'The endorsement of prominent business leaders', 'A focus on a bipartisan approach to major social and economic legislation', 'The creation of a strong alliance with the Republican Party', 'A focus on reducing government intervention in the economy']\",\"question\":\"Which of the following was a significant reason for the electoral success of Franklin Roosevelt's New Deal coalition during the 1930s?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}