{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7248e200-8480-5432-808d-303049a08078","task_key":"default--test--9045","task_revision_id":"1","upstream_id":"9045","short_description":"We were first able to accurately measure the diameter of Pluto from:","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"[\\\"Lunar-based observations made during NASA's Apollo missions\\\", 'The Voyager 2 spacecraft flyby in the 1980s', 'Observations made by the Chandra X-ray Observatory', 'brightness measurements made during mutual eclipses of Pluto and Charon', \\\"The Mars Rover's telescope observations of Pluto\\\", 'radar observations made by the Arecibo telescope', 'Observations made by the Spitzer Space Telescope', \\\"Hubble Space Telescope images that resolved Pluto's disk\\\", 'a New Horizons flyby in the 1990s', 'Images captured by the Kepler Space Telescope']\",\"question\":\"We were first able to accurately measure the diameter of Pluto from:\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}