{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a51916c-51b3-5b41-beda-ed67947fb928","task_key":"default--test--9724","task_revision_id":"1","upstream_id":"9724","short_description":"Optometrists use a linear scale of lens power, measured in diopters, to…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['An upright, real image about 20 cm behind her eyes', 'An upright, virtual image about 5 cm behind her eyes', 'An inverted, virtual image about 20 cm in front of her eyes', 'An inverted, real image about 20 cm behind her eyes', 'An inverted, real image about 20 cm in front of her eyes', 'An inverted, virtual image 5 cm behind her eyes', 'An upright, real image about 5 cm in front of her eyes', 'An upright, virtual image about 20 cm in front of her eyes', 'An upright, real image about 20 cm in front of her eyes', 'An inverted, real image about 5 cm in front of her eyes']\",\"question\":\"Optometrists use a linear scale of lens power, measured in diopters, to prescribe corrective lenses. Sufferers of myopia, or nearsightedness, have difficulty resolving distant objects because the lenses in their eyes are too strong. If a myopic person has a prescription of -5 diopters, what image do her corrective lenses create for distant objects?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}