{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a94e5498-4aea-5641-9251-99c0a11472d6","task_key":"all--test--a94e5498-4aea-5641-9251-99c0a11472d6","task_revision_id":"1","upstream_id":"","short_description":"A 50-year-old woman comes to the office for the first time because of recurrent…","config":"all","split":"test","body":"{\"choices\":\"['Conversion disorder', 'Histrionic personality disorder', 'Occult medical disorder', 'Somatic symptom disorder']\",\"question\":\"A 50-year-old woman comes to the office for the first time because of recurrent abdominal pain. Review of her extensive medical chart, which she has brought with her, discloses that she has a long history of varying physical symptoms. Definitive causes for these symptoms have not been found despite extensive diagnostic studies, consultations with many physicians, and several surgical explorations. She gives dramatic and exaggerated descriptions of her present and past symptoms, and she makes conflicting statements about her history. She has been hospitalized at least 23 times since age 18 years. Which of the following is the most likely diagnosis?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}