{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"621026a8-834d-5fee-827d-39104daf284f","task_key":"all--test--621026a8-834d-5fee-827d-39104daf284f","task_revision_id":"1","upstream_id":"","short_description":"A 17-year-old boy is brought to the emergency department 30 minutes after being…","config":"all","split":"test","body":"{\"choices\":\"['Diazepam', 'Methamphetamine', 'Oxycodone', 'PCP (phencyclidine)']\",\"question\":\"A 17-year-old boy is brought to the emergency department 30 minutes after being found with a \\\"blank stare\\\" and flat facial expression at a party. His pulse is 72/min, and blood pressure is 104/68 mm Hg. He is sitting upright and appears catatonic. Physical examination shows rigidity. During the examination, he becomes hostile and attempts to assault the physician. This patient most likely ingested which of the following drugs?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}