{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e496ad92-4129-50a6-9157-c62f0ba208c1","task_key":"default--test--5079","task_revision_id":"1","upstream_id":"5079","short_description":"Which of the following policies did the New Labour government not pursue?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['supporting LEAs that appeared to be failing', 'abolition of student tuition fees', 'privatization of state schools', 'state regulation and control of the curriculum', 'requiring all schools to provide vocational training', 'implementing a national dress code for teachers', 'setting and streaming pupils by ability', 'introduction of league tables and competition among schools', 'eliminating standardized testing', 'parental choice of school']\",\"question\":\"Which of the following policies did the New Labour government not pursue?\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}