{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"584cb4a6-dda0-5317-b7a3-80ba307af225","task_key":"all--test--584cb4a6-dda0-5317-b7a3-80ba307af225","task_revision_id":"1","upstream_id":"","short_description":"A 15-year-old boy is brought to the emergency department by his parents because…","config":"all","split":"test","body":"{\"choices\":\"['Facial nerve', 'Frontal sinus', 'Mastoid sinus', 'Olfactory nerve']\",\"question\":\"A 15-year-old boy is brought to the emergency department by his parents because of a 2-hour history of confusion and agitation. He also has had fever, headache, stiff neck, and vomiting since he returned from summer camp 2 days ago. His parents say he does not use illicit drugs. On arrival, he is combative and there is evidence of hallucinations. His temperature is 40°C (104°F), pulse is 80/min, respirations are 17/min, and blood pressure is 100/70 mm Hg. A lumbar puncture is performed. Results of cerebrospinal fluid analysis show cysts and trophozoites. The most likely portal of pathogen entry into this patient's central nervous system is which of the following?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}