{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b866f4e-fc77-5865-b660-b7a7825a924e","task_key":"default--test--9785","task_revision_id":"1","upstream_id":"9785","short_description":"The extremes of temperature in New York, over a period of 50 years, differ by…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['58.7 C°', '60.0 C°', '52.3 C°', '48.9 C°', '80.0 C°', '76.4 C°', '70.2 C°', '64.5 C°', '62.0 C°', '55.6 C°']\",\"question\":\"The extremes of temperature in New York, over a period of 50 years, differ by 116 Fahrenheit degrees. Express this 50 years, differ by 116 Fahrenheit degrees. Express this range range in Celsius degrees. in Celsius degrees.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}