{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81c6c188-22c3-5a48-845e-fee68613150e","task_key":"all--test--81c6c188-22c3-5a48-845e-fee68613150e","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is the most likely benefit of periodic natural wildfires…","config":"all","split":"test","body":"{\"choices\":\"['The fires remove dead and decaying plant matter, reducing the risk of more intense and destructive fires.', 'The fires leach nutrients from the soil‚ preventing the germination of plants that might compete with native species.', 'The fires drive off herbivores whose plant-based diets reduce the amount of vegetation.', 'The fires dry out the soil and decrease the chance of flooding after heavy rains.']\",\"question\":\"Which of the following is the most likely benefit of periodic natural wildfires in some forest ecosystems?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}