{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84a4e0d1-6f04-5d5c-91f6-1b0eec2a2233","task_key":"all--test--84a4e0d1-6f04-5d5c-91f6-1b0eec2a2233","task_revision_id":"1","upstream_id":"","short_description":"Why are there no impact craters on the surface of Io?","config":"all","split":"test","body":"{\"choices\":\"['Io did have impact craters but they have all been buried in lava flows.', 'It is too small to have been hit during the Late Heavy Bombardment', \\\"Jupiter's strong gravity attracted the planetesimals more strongly than Io and thus none landed on its surface.\\\", \\\"Any craters that existed have been eroded through the strong winds on Io's surface.\\\"]\",\"question\":\"Why are there no impact craters on the surface of Io?\",\"subject\":\"astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}