{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7bdfb4e2-86a1-55f7-a365-da42a0d0416c","task_key":"default--test--6689","task_revision_id":"1","upstream_id":"6689","short_description":"A 50-year-old woman comes to the office for the first time because of recurrent…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Somatic symptom disorder', 'Generalized anxiety disorder', 'Dependent personality disorder', 'Borderline personality disorder', 'Munchausen syndrome', 'Conversion disorder', 'Schizophrenia', 'Occult medical disorder', 'Narcissistic personality disorder', 'Histrionic personality disorder']\",\"question\":\"A 50-year-old woman comes to the office for the first time because of recurrent abdominal pain. Review of her extensive medical chart, which she has brought with her, discloses that she has a long history of varying physical symptoms. Definitive causes for these symptoms have not been found despite extensive diagnostic studies, consultations with many physicians, and several surgical explorations. She gives dramatic and exaggerated descriptions of her present and past symptoms, and she makes conflicting statements about her history. She has been hospitalized at least 23 times since age 18 years. Which of the following is the most likely diagnosis?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}