{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a5f4ca4-ca7e-5519-b494-fee2bdbc293b","task_key":"all--test--3a5f4ca4-ca7e-5519-b494-fee2bdbc293b","task_revision_id":"1","upstream_id":"","short_description":"A 67-year-old woman has had fatigue, dry skin, brittle hair, swelling of the…","config":"all","split":"test","body":"{\"choices\":\"['Chronic lymphocytic thyroiditis (Hashimoto disease)', 'Defect in thyroxine (T4) biosynthesis', 'Graves disease', 'Multinodular goiter']\",\"question\":\"A 67-year-old woman has had fatigue, dry skin, brittle hair, swelling of the ankles, and cold intolerance for 1 year; she has gained 9 kg (20 lb) during this period. Her pulse is 55/min, and blood pressure is 150/90 mm Hg. She appears lethargic. Examination shows dry skin and a nontender thyroid gland that is enlarged to two times its normal size. There is mild edema of the ankles bilaterally. The relaxation phase of the Achilles reflex is greatly prolonged. Which of the following is the most likely diagnosis?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}