{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"91a5f905-4354-5884-9c4b-da73286c7545","task_key":"default--test--9256","task_revision_id":"1","upstream_id":"9256","short_description":"What do meteorites reveal about the solar system?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['They reveal that the solar system was once part of a larger galaxy.', 'They reveal that all planets in the solar system once had life.', 'They reveal that the age of the solar system is approximately 4.6 billion years.', 'They reveal that the solar system is expanding at a constant rate.', 'They reveal that meteorites are the remnants of a fifth terrestrial planet.', 'They reveal that the solar system once contained 10 planets.', 'They reveal that meteorites are much older than the comets and planets.', 'They reveal that the solar system has always consisted of only 8 planets.', 'They reveal that the early solar system consisted mostly of hydrogen and helium gas.', 'They reveal that the solar system is actually much younger than previously thought.']\",\"question\":\"What do meteorites reveal about the solar system?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}