{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38f898e6-890a-56be-a2bd-d5931d9c8e5f","task_key":"default--test--8553","task_revision_id":"1","upstream_id":"8553","short_description":"A school committee member is lobbying for an increase in the gasoline tax to…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['All teachers in the county school system', 'All residents in the local newspaper distribution area', 'All county residents with children in the county school system', 'All parents of children in the county school system', 'All school committee members', 'All county residents with school-aged children', 'All county residents who pay gasoline tax', 'All county residents', 'All school-aged children', 'All local newspaper readers']\",\"question\":\"A school committee member is lobbying for an increase in the gasoline tax to support the county school system. The local newspaper conducted a survey of county residents to assess their support for such an increase. What is the population of interest here?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}