{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49fd9685-6492-5976-8540-0695ee645942","task_key":"all--test--49fd9685-6492-5976-8540-0695ee645942","task_revision_id":"1","upstream_id":"","short_description":"Which of the following best helps explain why volcanoes tend to form along…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Rock material carried deep below Earth's surface results in material melting and rising to the surface.\\\", 'Two plates pull away from each other, and magma oozes out.', 'The friction resulting from the collision of two continents causes rock to melt.', 'There is a row of hot spots at these boundaries.']\",\"question\":\"Which of the following best helps explain why volcanoes tend to form along subduction zones?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}