{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac659a3b-111f-53ec-ab77-b3f622fdc567","task_key":"default--test--10210","task_revision_id":"1","upstream_id":"10210","short_description":"Why is the Mars Exploration Rover Spirit currently tilted towards the north?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Because it’s in the northern hemisphere where it is winter now.', 'Because it’s in the southern hemisphere where it is winter now.', \\\"Because it's avoiding a sand storm.\\\", 'Because it’s climbing up a big hill.', \\\"Because it's trying to get a better signal reception.\\\", \\\"Because it's stuck in a Martian crater.\\\", \\\"Because it's positioned near the Martian equator where the sun is always in the north.\\\", \\\"Because it's programmed to tilt north during specific tasks.\\\", \\\"Because it's attempting to maximize solar power generation.\\\", 'Because one of its wheels broke.']\",\"question\":\"Why is the Mars Exploration Rover Spirit currently tilted towards the north?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}