{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81cc331d-82f8-5c7c-bfe4-ebc163d9315f","task_key":"all--test--81cc331d-82f8-5c7c-bfe4-ebc163d9315f","task_revision_id":"1","upstream_id":"","short_description":"When a wave moves from shallow water to deep water, the","config":"all","split":"test","body":"{\"choices\":\"['frequency decreases, the wavelength increases, and the speed increases', 'frequency does not change, the wavelength increases, and the speed increases', 'frequency does not change, the wavelength decreases, and the speed decreases', 'frequency does not change, the wavelength increases, and the speed decreases']\",\"question\":\"When a wave moves from shallow water to deep water, the\",\"subject\":\"conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}