{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f61d4d77-7929-594c-9cba-dc25927e4942","task_key":"default--test--9985","task_revision_id":"1","upstream_id":"9985","short_description":"What is the refracting power of a concave mirror of radius 20 cm when the…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['30 diopters in air, 39.99 diopters in water', '25 diopters in air, 33.325 diopters in water', '20 diopters in air, 26.66 diopters in water', '12.5 diopters in air, 16.6625 diopters in water', '17.5 diopters in air, 23.325 diopters in water', '2.5 diopters in air, 3.3325 diopters in water', '15 diopters in air, 20 diopters in water', '10 diopters in air, 13.33 diopters in water', '7.5 diopters in air, 10 diopters in water', '5 diopters in air, 6.665 diopters in water']\",\"question\":\"What is the refracting power of a concave mirror of radius 20 cm when the reflecting surface is in contact with (a) air (n = 1) and (b) water (n = 1.33; n denotes index of refraction)?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}