{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b16a06e9-5182-5678-b0c6-8e9831e53171","task_key":"default--test--2363","task_revision_id":"1","upstream_id":"2363","short_description":"What are the symptoms of a paranoiddisorder ?Discuss the threetypes of this…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The symptoms of paranoid disorder are persistent, persecutory delusions or delusions of jealousy. The three types of paranoid disorders are paranoia, shared paranoia, and paranoid state.', 'Symptoms include hallucinations and mood swings. The three types are delusional disorder, shared delusion, and paranoia', 'Symptoms are delusions and hallucinations. Types include paranoid schizophrenia, paranoid personality disorder, and delusional disorder.', 'Symptoms are distrust and communication difficulties. The types include paranoid anxiety, paranoid personality disorder, and paranoid psychosis.', 'The symptoms include obsessive thoughts and compulsive behaviors. The types are shared paranoia, paranoid personality disorder, and paranoia', 'Symptoms are anxiety and depression. Types include shared paranoia, paranoia, and paranoid personality disorder', 'The symptoms include chronic anxiety and mistrust. The types are paranoid schizophrenia, paranoia, and paranoid psychosis.', 'The symptoms include extreme suspicion and social isolation. The types are delusional disorder, paranoia, and paranoid anxiety disorder.', 'Symptoms include persistent fear and social withdrawal. The types are paranoid psychosis, shared paranoid disorder, and paranoid depression.', 'The symptoms of paranoid disorder are delusions of grandeur and auditory hallucinations. The three types are paranoid personality disorder, paranoid schizophrenia, and paranoia.']\",\"question\":\"What are the symptoms of a paranoiddisorder ?Discuss the threetypes of this disorder.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}