{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03f91609-3b3f-5699-b0e7-505ee6976060","task_key":"default--test--9165","task_revision_id":"1","upstream_id":"9165","short_description":"What is not true of Jupiter's magnetic field?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"[\\\"it is directly influenced by the Sun's magnetic field\\\", 'it switches polarity every 11 years', \\\"it is about 20000 times stronger than Earth's magnetic field\\\", \\\"it is weaker on the planet's dark side\\\", \\\"it causes spectacular auroral displays at Jupiter's north and south poles\\\", 'it extends beyond the orbit of Saturn', 'it traps charged particles from Io\\\\'s volcanoes in a \\\"plasma torus\\\" around the planet', \\\"it is responsible for Jupiter's rings\\\", \\\"it is generated by the fast rotation of Jupiter's metallic hydrogen inner core\\\", \\\"it has a similar structure to Earth's magnetic field\\\"]\",\"question\":\"What is not true of Jupiter's magnetic field?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}