{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d17b0261-661e-5004-8649-296cac9c8d12","task_key":"default--test--9670","task_revision_id":"1","upstream_id":"9670","short_description":"A plane mirror is suspended vertically at the center of a large sphericalflask…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['40 inches from the original object', '28 inches from the original object', '9 inches from the original object', '26 inches from the original object', '21 inches from the original object', '35 inches from the original object', '30 inches from the original object', '17.5 inches from the original object', '15 inches from the original object', '12 inches from the original object']\",\"question\":\"A plane mirror is suspended vertically at the center of a large sphericalflask filled with water (the index of refraction for wateris 1.33). The diameter of the flask is 10 inches. An observerwhose eye is 35 inches from the mirror sees an imageof his own eye. Where is the image seen?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}