{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fdc0ff61-18bb-5f6c-8d35-3edfcea05a5d","task_key":"default--test--2731","task_revision_id":"1","upstream_id":"2731","short_description":"Name and describe the seven symptom types of neuroses.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['General anxiety, Specific phobias, Major depressive episodes, Fugue states, Compulsive rituals, Somatic symptom disorder, Psychoanalytic disorder', 'Anxiety, Phobia, Depression, Dissociation, Obsession, Conversion, Neurotic', 'Fear response, Aviophobia (fear of flying), Seasonal affective disorder, Multiple personality disorder, Fixation, Pseudoneurological symptom, Adjustment disorder', 'Anxiety disorder, Fear disorder, Depression disorder, Amnesia, Obsession disorder, Conversion disorder, Psychoneurotic disorder', 'Anxiety attack, Phobic disorder, Clinical depression, Dissociative disorder, Obsessive-Compulsive Disorder, Conversion disorder, Neurotic disorder', 'Stress reaction, Panic reaction, Mood swings, Memory loss, Compulsive behavior, Somatization, Psychosomatic reaction', 'Nervousness, Acrophobia (fear of heights), Cyclothymia, Retrograde amnesia, Perfectionism, Functional neurological disorder, Behavioral disorder', 'Panic syndrome, Agoraphobia, Bipolar conditions, Identity confusion, Ritualistic obsession, Body dysmorphic reaction, Psychophysiological reaction', 'Anxiety reaction, Phobic reactions, Depressive reactions, Dissociative reactions, Obsessive and compulsive reactions, Conversion reactions, Psychoneurotic reaction', 'Social anxiety, Claustrophobia, Dysthymia, Selective amnesia, Habit disorder, Hypochondriasis, Personality disorder']\",\"question\":\"Name and describe the seven symptom types of neuroses.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}