{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aeb8d839-4679-5cfa-bbdc-d3542ca621b6","task_key":"default--test--2321","task_revision_id":"1","upstream_id":"2321","short_description":"Describe the four major types of conduct disorders and the main characteristics…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Phobias, PTSD, OCD, panic disorder', 'Oppositional defiant disorder, intermittent explosive disorder, kleptomania, pyromania', 'Attention deficit disorder, conduct disorder, separation anxiety disorder, reactive attachment disorder', 'Depression, anxiety, bipolar disorder, schizophrenia', 'Somatic symptom disorder, dissociative identity disorder, conversion disorder, factitious disorder', 'Narcissistic personality disorder, borderline personality disorder, histrionic personality disorder, antisocial personality disorder', 'ADHD, autism, dyslexia, dyspraxia', 'Learning disabilities, language disorders, motor skills disorders, cognitive disorders', 'Eating disorders, sleep-wake disorders, impulse-control disorders, neurocognitive disorders', 'Antisocial reactions, dyssocial reactions, addictions, and sexual deviations']\",\"question\":\"Describe the four major types of conduct disorders and the main characteristics of each.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}