{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47a51a95-fd81-53ac-8918-1243ac3398d8","task_key":"all--test--47a51a95-fd81-53ac-8918-1243ac3398d8","task_revision_id":"1","upstream_id":"","short_description":"A 47-year-old man comes to the physician 12 hours after the sudden onset of a…","config":"all","split":"test","body":"{\"choices\":\"['Cluster headache', 'Meningitis', 'Migraine', 'Subarachnoid hemorrhage']\",\"question\":\"A 47-year-old man comes to the physician 12 hours after the sudden onset of a severe occipital headache and stiff neck. He has not had any other symptoms and has no history of severe headache. He has hypertension and gastroesophageal reflux disease. Current medications include hydrochlorothiazide and ranitidine. He is oriented to person, place, and time. His temperature is 36.7°C (98.1°F), pulse is 100/min, and blood pressure is 160/90 mm Hg. Range of motion of the neck is decreased due to pain. Neurologic examination shows no focal findings. Which of the following is the most likely diagnosis?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}