{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86611d5d-3e0e-54f8-a77f-7d4c2f5f7186","task_key":"all--test--86611d5d-3e0e-54f8-a77f-7d4c2f5f7186","task_revision_id":"1","upstream_id":"","short_description":"Ethnographic evidence supports which of the following connections between…","config":"all","split":"test","body":"{\"choices\":\"['Only agriculturalists can produce the food surplus necessary for economic inequality.', 'Some resource-rich environments allow for the creation of surpluses that can lead to economic inequalities and social complexity.', 'Economic inequalities are the foundation of social complexity and entirely dependent on agriculture for the production of surplus food.', 'The production of surplus food is not important in the development of economic inequalities or social complexity.']\",\"question\":\"Ethnographic evidence supports which of the following connections between surplus food and social complexity?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}