{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c751e155-602a-5787-b56a-e2b95f72d03f","task_key":"default--test--5429","task_revision_id":"1","upstream_id":"5429","short_description":"Which of the following statements is true regarding delegates to presidential…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Delegates are often chosen at random from the general population.', 'Delegates often switch party affiliations during the convention.', 'Delegates typically have little interest in politics.', 'The majority of delegates are not affiliated with any political party.', 'Delegates are generally more likely to register as third party voters at some point.', 'The majority of the general population is more educated than the delegates.', 'Delegates are typically less educated than the general population.', 'Delegates usually have no influence on the presidential nomination process.', 'Typically, delegates have a lower level of political knowledge than the general population.', 'Typically, the general population is less ideological than the delegates.']\",\"question\":\"Which of the following statements is true regarding delegates to presidential nomination conventions?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}