{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4136eac8-86f6-52b8-9660-bfcd06bd41bb","task_key":"all--test--4136eac8-86f6-52b8-9660-bfcd06bd41bb","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is evidence of new uses for plants during the Middle…","config":"all","split":"test","body":"{\"choices\":\"['the use of seeds as a natural insecticide in Altamira cave in southwestern Europe', 'plant materials such as flowers in Aurignacian burials in southern France', 'plant material lining the floor of Sibudu Cave in South Africa', 'the use of wood in the Gravettian tradition to make spear-throwers and canoes']\",\"question\":\"Which of the following is evidence of new uses for plants during the Middle Stone Age?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}