{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2276136-1261-54ed-9966-103dd3155d04","task_key":"default--test--7923","task_revision_id":"1","upstream_id":"7923","short_description":"At West Elementary School, there are 20 more girls than boys. If there are 180…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['divide 20 by 180', 'add 180 to 20', 'multiply 180 by 20', 'multiply 180 by 2', 'divide 180 by 20', 'subtract 20 from 20', 'add 20 to 180', 'subtract 20 from 180', 'multiply 20 by 180', 'subtract 180 from 20']\",\"question\":\"At West Elementary School, there are 20 more girls than boys. If there are 180 girls, how can you find the number of boys?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}