{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68ac2598-124a-57a1-af52-d956a1fdf178","task_key":"default--test--10298","task_revision_id":"1","upstream_id":"10298","short_description":"The sky is blue because","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"[\\\"the Sun's rays pass through a blue layer of the atmosphere.\\\", 'the Sun mainly emits blue light.', 'molecules scatter blue light more effectively than red light.', 'the atmosphere absorbs mostly blue light.', 'molecules scatter red light more effectively than blue light.', 'the Earth is mostly covered by water which reflects blue light.', 'the sky reflects the color of the ocean.', 'the ozone layer emits a blue glow.', \\\"clouds are actually blue, but appear white because of the Sun's brightness.\\\", \\\"blue is the color of space and we're seeing through the atmosphere.\\\"]\",\"question\":\"The sky is blue because\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}