{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99a2e49a-72fe-5b87-aa8d-4c6729cbd08f","task_key":"all--test--99a2e49a-72fe-5b87-aa8d-4c6729cbd08f","task_revision_id":"1","upstream_id":"","short_description":"Which of the following sequences of events is consistent with the presence of a…","config":"all","split":"test","body":"{\"choices\":\"['A volcano erupted, sending lava out over a layer of shale. The lava cooled and hardened into sandstone.', 'The water level of a large lake lowered. A beach then formed where previously there had been lake bottom.', 'One-celled organisms developed a colony on the seafloor. Shells made by these organisms accumulated and lithified, forming the sandstone.', 'Mud was deposited and lithified. Subsequent contact metamorphism resulted in localized recrystallization of the shale into sandstone.']\",\"question\":\"Which of the following sequences of events is consistent with the presence of a layer of sandstone in bedrock just above a layer of shale?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}