{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe51aafb-1039-5163-baca-126ddae44ed5","task_key":"default--test--9208","task_revision_id":"1","upstream_id":"9208","short_description":"A man stands with his hands to his sides on a frictionless platform that is…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The man starts to jump and land back repeatedly on the platform', 'The man lies down on the platform', 'The man spins in the opposite direction of the platform', 'The man does a handstand on the platform', 'The man thrusts his arms out away from his body, and then quickly brings his arms back to his side again.', 'The man catches a baseball thrown to him by a friend.', 'The man thrusts his arms out away from his body', 'The man jumps straight up in the air and lands back on the platform.', 'The man starts to walk in circles on the platform', 'The man sits down on the platform']\",\"question\":\"A man stands with his hands to his sides on a frictionless platform that is rotating. Which of the following could change the angular momentum of the man-platform system?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}