{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d82314e9-80d4-5a84-9e95-9a1a96b38868","task_key":"default--test--7745","task_revision_id":"1","upstream_id":"7745","short_description":"A customer is not certain if the advertised width of a 48-inch television is…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['2', '6', '5', '9', '4', '1', '3', '7', '10', '8']\",\"question\":\"A customer is not certain if the advertised width of a 48-inch television is along the horizontal length of the screen or along the diagonal of the screen. If the ratio of the length to the height of the television screen is 5 to 3, how much shorter, to the nearest inch, will the horizontal length of the television screen be if the measure is made along the diagonal of the television screen rather than along the horizontal length?\",\"src\":\"ori_mmlu-high_school_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}