{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aed0cf81-e0b3-5ad2-88d1-1aefce7ee9f5","task_key":"default--test--5589","task_revision_id":"1","upstream_id":"5589","short_description":"If no presidential candidate receives a majority of the electoral college vote,…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The candidate who wins the majority of votes in the state with the most electoral votes', 'The candidate with the second highest number of electoral votes', 'The candidate who wins the most votes in the electoral college', 'The candidate who wins the majority of votes in the House', 'Whomever is selected by the United States Congress', 'The candidate chosen by a national popular vote after the electoral college deadlock', 'The candidate who wins the popular national vote', 'The incumbent vice president of the United States', 'The candidate who wins the majority of votes in the Senate', 'Whomever is selected by the United States Supreme Court']\",\"question\":\"If no presidential candidate receives a majority of the electoral college vote, then who becomes the president?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}