{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3089cdc0-a749-58bc-bb51-87354613a355","task_key":"all--test--3089cdc0-a749-58bc-bb51-87354613a355","task_revision_id":"1","upstream_id":"","short_description":"Which of the following describes a symbiotic relationship?","config":"all","split":"test","body":"{\"choices\":\"[\\\"A tapeworm feeds off of its host's nutrients causing the host to lose large amounts of weight.\\\", 'Certain plants grow on trees in order to gain access to sunlight, not affecting the tree.', 'Remora fish eat parasites off of sharks; the sharks stay free of parasites, and the remora fish are protected from predators.', 'Meerkats sound alarm calls to warn other meerkats of predators.']\",\"question\":\"Which of the following describes a symbiotic relationship?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}