{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dac4b821-48db-55b8-b772-049f66863ba9","task_key":"all--test--dac4b821-48db-55b8-b772-049f66863ba9","task_revision_id":"1","upstream_id":"","short_description":"In rare cases, the residue of blood on ancient stone tools can:","config":"all","split":"test","body":"{\"choices\":\"['prevent archaeologists from doing their research, since only forensic anthropologists can study bloody tools.', 'be used to identify the species of animals killed and likely eaten.', 'be used to predict the types of weapons that were not used for hunting.', 'none of the above; blood does not preserve and has never been identified on stone tools.']\",\"question\":\"In rare cases, the residue of blood on ancient stone tools can:\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}