{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b030aa96-0c95-503b-a7ef-8b5847020f3f","task_key":"default--test--5588","task_revision_id":"1","upstream_id":"5588","short_description":"Which of the following statements does NOT accurately describe voting behavior…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Registered voters between the ages of 35 and 45 are more likely to vote than are those under the age of 21.', 'A registered voter who has attained his or her General Educational Development (GED) is less likely to vote than a high school dropout.', 'Registered voters are more likely to vote in general elections than they are in primary elections.', 'More women than men have voted in every presidential election since 1980.']\",\"question\":\"Which of the following statements does NOT accurately describe voting behavior in the United States?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}