{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"072016a4-8fee-5338-bd26-886637b5e08e","task_key":"default--test--9366","task_revision_id":"1","upstream_id":"9366","short_description":"How are planetary rings made?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['From accretion within the solar nebula at the same time the planets formed', 'From fragments of planets ejected by impacts', 'From dust grains that escape from passing comets', \\\"From the remnants of a planet's atmosphere\\\", 'From the gravitational pull of nearby planets', 'From the dismantling of small moons by impacts', 'From the fusion of space dust and meteorites']\",\"question\":\"How are planetary rings made?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}