{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9b036fc-8214-541a-815b-cf31a10d7799","task_key":"default--test--10055","task_revision_id":"1","upstream_id":"10055","short_description":"Physics students are checking the constant acceleration equations of kinematics…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Air resistance increases the acceleration of the ball.', 'The ball experienced magnetic interference during the fall.', \\\"The ball's velocity decreased due to collision with another object.\\\", 'The gravitational pull fluctuates during the fall.', 'The acceleration of gravity varies as the ball is falling.', 'The acceleration of the balls varies with the velocity.', 'The ball changes its shape while falling.', \\\"The ball's mass increased during the fall.\\\"]\",\"question\":\"Physics students are checking the constant acceleration equations of kinematics by measuring the velocity of a tennis ball that is dropped and falls 6 meters and then passes through a photogate. The predicted velocity is 20% above the velocity measured by the photogate. Which of the following best describes the cause of the large percent difference?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}