{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39e150b3-342a-5052-834a-1d149e3c0564","task_key":"all--test--39e150b3-342a-5052-834a-1d149e3c0564","task_revision_id":"1","upstream_id":"","short_description":"A 60-year-old man had a total thyroidectomy and excision of enlarged left…","config":"all","split":"test","body":"{\"choices\":\"['Babinski sign present bilaterally', 'Chvostek sign', 'Deviation of the tongue to the left side', 'A drooping left shoulder']\",\"question\":\"A 60-year-old man had a total thyroidectomy and excision of enlarged left jugular lymph nodes for follicular carcinoma. The operation was uncomplicated. He is receiving intravenous 5% dextrose and 0.45% saline with potassium. Twelve hours after the operation he develops circumoral numbness and paresthesias in his fingertips, and he becomes very anxious. Vital signs are temperature 37.6°C (99.7°F), pulse 90/min, respirations 16/min, and blood pressure 140/90 mm Hg. Physical examination discloses a dry neck dressing and no stridor. Extremities are warm, with brisk capillary refill time. Additional physical examination is most likely to show which of the following?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}