{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"716497ea-3f2e-5e95-ad27-23931051fbc5","task_key":"default--test--10332","task_revision_id":"1","upstream_id":"10332","short_description":"If a star that is 20 light-years from Earth explodes, gravitational waves from…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['10 years, as gravitational waves travel at double the speed of light', '5 years, as gravitational waves travel at four times the speed of light', '40 years, as gravitational waves travel at half the speed of light', 'Instantly, as gravitational waves travel at infinite speed', '20 years.', 'None of these.', '100 years, as gravitational waves travel at a fifth the speed of light', 'more than 20 years.', 'less than 20 years.', 'It would never reach Earth']\",\"question\":\"If a star that is 20 light-years from Earth explodes, gravitational waves from the explosion would reach Earth in\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}