{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"557a4c4d-bf8b-5a02-b63c-4bebbccbbc7e","task_key":"default--test--9477","task_revision_id":"1","upstream_id":"9477","short_description":"Why is the sky blue?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"[\\\"Because the Earth's magnetic field causes the sky to appear blue.\\\", 'Because the human eye is more sensitive to blue light.', \\\"Because the Earth's atmosphere contains a large amount of blue pigment.\\\", 'Because the atmosphere preferentially scatters short wavelengths.', 'Because blue is the most abundant color in the color spectrum.', \\\"Because the molecules that compose the Earth's atmosphere have a blue-ish color.\\\", 'Because the ozone layer reflects blue light.', 'Because the sun emits blue light more than any other color.', \\\"Because the Earth's atmosphere preferentially absorbs all other colors.\\\", \\\"Because the sky reflects the color of the Earth's oceans.\\\"]\",\"question\":\"Why is the sky blue?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}