{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5bd441b0-17b1-567b-8885-5367cf45430e","task_key":"default--test--9261","task_revision_id":"1","upstream_id":"9261","short_description":"Why can't we see the Sun's corona except during total solar eclipses?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"[\\\"We can't see magnetic fields\\\", \\\"The corona is blocked by the Earth's atmosphere\\\", \\\"The corona is hidden by the Sun's photosphere\\\", \\\"The corona is made up mostly of charged particles which aren't luminous.\\\", \\\"The Sun's corona is actually invisible to the human eye\\\", \\\"It's too diffuse\\\", 'The corona is only visible at night', \\\"It's too hot to emit visible light\\\", \\\"It's too far away from the Sun's surface\\\", \\\"It's much too cool to emit visible light\\\"]\",\"question\":\"Why can't we see the Sun's corona except during total solar eclipses?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}