{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e303bdfd-2586-5cc3-bcef-e9b4b6744fbb","task_key":"default--test--10099","task_revision_id":"1","upstream_id":"10099","short_description":"Planet X orbits its sun once every 100 Earth days and rotates in the prograde…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['There is no solar day because Planet X does not have a sun.', 'About 200 earth days', 'About 300 earth days', 'About 400 earth days', 'About 500 earth days', 'About 150 earth days', 'About 100 earth days', 'About 50 earth days', 'About 10 earth days', 'There is no solar day Planet X is tidally locked']\",\"question\":\"Planet X orbits its sun once every 100 Earth days and rotates in the prograde direction around its axis with a sidereal day of 100 Earth days. How long is a solar day on Planet X?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}