{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"91329c99-dc21-52cc-b0c7-23527f47f9ca","task_key":"all--test--91329c99-dc21-52cc-b0c7-23527f47f9ca","task_revision_id":"1","upstream_id":"","short_description":"Pretend someone actually managed to dig a hole straight through the center of…","config":"all","split":"test","body":"{\"choices\":\"['It would fall to the center of the Earth and stop there.', 'It would fall through the hole to the other side, continue past the opposite side’s opening, and fly into space.', 'It would oscillate back and forth from one opening to the other indefinitely.', 'It would fall to the other side and stop there.']\",\"question\":\"Pretend someone actually managed to dig a hole straight through the center of the Earth all the way to the other side. If an object were dropped down that hole, which of the following would best describe its motion? Assume ideal conditions and that the object cannot be destroyed.\",\"subject\":\"high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}