{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3500e8b2-6ad7-52d7-86cd-3afdcba9a985","task_key":"default--test--10301","task_revision_id":"1","upstream_id":"10301","short_description":"If the Sun were suddenly replaced by a black hole of the same mass, it would…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The planets would be thrown out of the solar system.', 'The planets would rotate more slowly.', 'The planets would immediately be sucked into the black hole.', 'The planets would move directly toward the Sun.', 'The orbits would become circular.', 'The planets would move in spiral orbits.', 'The orbits would change based on the size of the planet.', 'The orbits would become more elliptical.', 'The planets would oscillate about their former elliptical orbits.', 'The orbits would remain unchanged.']\",\"question\":\"If the Sun were suddenly replaced by a black hole of the same mass, it would have a Schwarzschild radius of 3,000 m. What effect, if any, would this change have on the orbits of the planets?\",\"src\":\"ori_mmlu-college_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}