{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86895868-984c-5478-89dc-8628ad341331","task_key":"all--test--86895868-984c-5478-89dc-8628ad341331","task_revision_id":"1","upstream_id":"","short_description":"A man stands with his hands to his sides on a frictionless platform that is…","config":"all","split":"test","body":"{\"choices\":\"['The man catches a baseball thrown to him by a friend.', 'The man thrusts his arms out away from his body', 'The man thrusts his arms out away from his body, and then quickly brings his arms back to his side again.', 'The man jumps straight up in the air and lands back on the platform.']\",\"question\":\"A man stands with his hands to his sides on a frictionless platform that is rotating. Which of the following could change the angular momentum of the man-platform system?\",\"subject\":\"high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}