{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18ae9c73-25a9-5ffa-b052-837d4e5b2f17","task_key":"default--test--10096","task_revision_id":"1","upstream_id":"10096","short_description":"What is the Cassini division of Saturn's rings?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['a cluster of small moonlets within the ring system', 'the outermost ring of Saturn, composed of ice and rock particles', 'the widest ring of Saturn located between two large ring gaps', 'a large gap visible from Earth produced by an orbital resonance with the moon Mimas', 'a dark ring visible from Earth composed of dark dusty particles', \\\"a bright ring formed by a concentration of icy particles in Saturn's magnetic field\\\", \\\"a small gap in the ring system caused by the moon Titan's gravitational influence\\\", 'a narrow band of ringlets located outside the A Ring', 'the most opaque ring of Saturn made of highly reflective ice particles', \\\"a region of the ring system that's unaffected by Saturn's gravitational pull\\\"]\",\"question\":\"What is the Cassini division of Saturn's rings?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}