{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c44e8560-825f-55bf-be00-48a86253ae59","task_key":"all--test--c44e8560-825f-55bf-be00-48a86253ae59","task_revision_id":"1","upstream_id":"","short_description":"A well on the side of a hill penetrates the water table at an elevation of 550…","config":"all","split":"test","body":"{\"choices\":\"['The water table would be found at sea level.', 'The water table would be found at an elevation lower than 550 feet above sea level but above sea level.', 'The water table would be found at an elevation higher than 550 feet above sea level but many feet below the land surface.', 'The water table would intersect with the land surface at the top of the hill, forming a spring.']\",\"question\":\"A well on the side of a hill penetrates the water table at an elevation of 550 feet above sea level. If someone plans to drill on the top of the hill, where would the water table most likely be found?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}