{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ae95a8c-ad4f-5b73-b92a-6112463e73b4","task_key":"default--test--10296","task_revision_id":"1","upstream_id":"10296","short_description":"Mars has an atmosphere that is almost entirely carbon dioxide.Why isn't there a…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Mars does not have enough internal heat to drive the greenhouse effect', 'the atmosphere on Mars is too thin to trap a significant amount of heat', 'There actually is a strong greenhouse effect and Mars would be 35oC colder than it is now without it.', 'The Martian soil absorbs all the heat, negating the greenhouse effect', 'The greenhouse effect on Mars is counteracted by its rapid rotation', 'Mars lacks the necessary amount of water vapor for a greenhouse effect', \\\"The high levels of carbon dioxide in Mars' atmosphere actually cool the planet down\\\", 'Mars is too far from the sun for the greenhouse effect to take place', \\\"Mars' poles are covered in ice which reflects sunlight and prevents the greenhouse effect.\\\", 'the greenhouse effect requires an ozone layer which Mars does not have']\",\"question\":\"Mars has an atmosphere that is almost entirely carbon dioxide.Why isn't there a strong greenhouse effect keeping the planet warm?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}