{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1f604372-9562-5c66-89fb-c64083b56128","task_key":"default--test--2372","task_revision_id":"1","upstream_id":"2372","short_description":"Discuss affective disorders as they are described in DSM-III.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Affective disorders are always chronic and long lasting', 'Affective disorders in DSM-III are categorized solely based on the duration of symptoms, irrespective of mood state', 'Affective disorders in DSM-III are only classified according to the predominant mood displayed by the patient', 'Affective disorders are characterized by a disturbance of mood accompanied by related symptoms, and can be categorized into depressed, manic, or bipolar. They are further divided into episodic, chronic, and atypical affective disorders in DSM III.', 'Affective disorders are a group of disorders where mood instability is due to external environmental factors, as per DSM-III', 'Affective disorders involve only feelings of extreme happiness or mania', 'Affective disorders are not recognized in DSM-III and were introduced in later versions', 'Affective disorders are defined by a single episode of depression with no recurrence', 'Affective disorders include only depressive episodes without episodes of mania or hypomania', 'Affective disorders are classified based on cognitive impairments rather than mood disturbances']\",\"question\":\"Discuss affective disorders as they are described in DSM-III.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}