{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3250b408-e8e5-57ec-a93b-5763dea520e3","task_key":"all--test--3250b408-e8e5-57ec-a93b-5763dea520e3","task_revision_id":"1","upstream_id":"","short_description":"Mimicry is a strategy that has evolved through natural selection to increase the…","config":"all","split":"test","body":"{\"choices\":\"[\\\"A type of millipede that is toxic to a toad is permanently avoided by the toad following the toad's initial attempt to consume it.\\\", 'A moth exhibits false eyes at its tail end in order to disorient predators.', 'A moth exhibits nearly identical coloration to that of a stinging bee.', 'A ground-nesting gull chick displays a coloration pattern that is nearly indistinguishable from its surroundings.']\",\"question\":\"Mimicry is a strategy that has evolved through natural selection to increase the fitness of organisms to their environment. Which of the following represents a form of Batesian mimicry?\",\"subject\":\"college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}