{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45a4f053-597c-5a34-8123-d435603a121f","task_key":"default--test--3419","task_revision_id":"1","upstream_id":"3419","short_description":"Briefly discuss some abnormalities of the respiratory tract.","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Influenza, chickenpox, and measles', \\\"Multiple sclerosis, Parkinson's disease, and Alzheimer's disease\\\", 'Arthritis, osteoporosis, and fibromyalgia', 'Hypertension, heart disease, and diabetes', 'Chronic obstructive pulmonary disease (COPD), cystic fibrosis, and pleural effusion', \\\"Gastroesophageal reflux disease (GERD), peptic ulcers, and Crohn's disease\\\", 'Astigmatism, glaucoma, and cataracts', 'Tuberculosis, laryngitis, bronchitis, pneumonia, hay fever, allergic asthma, lung cancer, emphysema, and respiratory distress syndrome are some abnormalities of the respiratory tract.', 'Sleep apnea, sarcoidosis, and pulmonary embolism']\",\"question\":\"Briefly discuss some abnormalities of the respiratory tract.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}