{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d8e62a5-40b6-5597-a506-24ff24f18cce","task_key":"default--test--5993","task_revision_id":"1","upstream_id":"5993","short_description":"Which of the following is LEAST associated with the development of alluvial…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['A deep cover of loose materials', 'Abundance of wildlife', 'High levels of precipitation', 'Deep dissection and erosion', 'The presence of glaciers', 'Mountainous terrain', 'Frequent flooding', 'A predominance of gentle slopes', 'The presence of large bodies of water']\",\"question\":\"Which of the following is LEAST associated with the development of alluvial landscapes?\",\"src\":\"ori_mmlu-high_school_geography\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}