{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"718a2463-21f6-5abb-8eb1-daa2ac8d38a6","task_key":"default--test--6748","task_revision_id":"1","upstream_id":"6748","short_description":"A 15-year-old boy is brought to the emergency department by his parents because…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Mastoid sinus', 'Frontal sinus', 'Nasal cavity', 'Olfactory nerve', 'Vertebral canal', 'Oral cavity', 'Optic nerve', 'Eustachian tube', 'Auditory canal', 'Facial nerve']\",\"question\":\"A 15-year-old boy is brought to the emergency department by his parents because of a 2-hour history of confusion and agitation. He also has had fever, headache, stiff neck, and vomiting since he returned from summer camp 2 days ago. His parents say he does not use illicit drugs. On arrival, he is combative and there is evidence of hallucinations. His temperature is 40°C (104°F), pulse is 80/min, respirations are 17/min, and blood pressure is 100/70 mm Hg. A lumbar puncture is performed. Results of cerebrospinal fluid analysis show cysts and trophozoites. The most likely portal of pathogen entry into this patient's central nervous system is which of the following?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}