{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60fe168e-5295-56d5-b064-ffe192c34e57","task_key":"default--test--9992","task_revision_id":"1","upstream_id":"9992","short_description":"The nebular theory of the formation of the solar system successfully predicts…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The presence of moons around several planets', 'Planets orbit around the Sun in nearly circular orbits in a flattened disk.', 'the equal number of terrestrial and jovian planets', \\\"The existence of the Sun's magnetic field\\\", 'The different chemical compositions of the planets', 'The presence of water on Earth', 'The distribution of meteorites across the solar system', 'The rotation of the planets on their axes', 'the craters on the Moon', 'asteroids Kuiper-belt comets and the Oort cloud']\",\"question\":\"The nebular theory of the formation of the solar system successfully predicts all but one of the following. Which one does the theory not predict?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}