{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c05a2e34-8939-56eb-b3e4-1bcc2ec8f3a3","task_key":"default--test--2335","task_revision_id":"1","upstream_id":"2335","short_description":"The diagnosis of Alzheimer's disease can be most accurately verified by","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Blood test showing elevated levels of certain proteins', 'Electroencephalogram (EEG) showing abnormal brain activity', 'ruling out other etiologies through a comprehensive psychodiagnostics workup', 'neuropsychological testing showing the expected profile of deficits and spared functions', 'Physical examination and patient history', 'brain biopsy', 'computerized tomography (CT) or magnetic resonance imaging scan showing cerebral atrophy', 'Spinal tap or lumbar puncture', 'Positron emission tomography (PET) scan showing high glucose levels', \\\"Genetic testing for Alzheimer's related genes\\\"]\",\"question\":\"The diagnosis of Alzheimer's disease can be most accurately verified by\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}