{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"078dce7d-6080-5afd-9fbb-39be434aeeb2","task_key":"default--test--9874","task_revision_id":"1","upstream_id":"9874","short_description":"Which factor s most important in determining the history of volanism and…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['size of the planet', 'proximity to other planets', 'distance from the sun', 'presence of water', 'rotation period', 'age of the planet', 'number of moons', 'presence of an atmosphere', \\\"composition of the planet's core\\\", 'magnetic field strength']\",\"question\":\"Which factor s most important in determining the history of volanism and tectonism on a planet?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}