{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c97c9d71-9be3-5e22-bf7a-2aaff92943de","task_key":"default--test--3514","task_revision_id":"1","upstream_id":"3514","short_description":"Briefly discuss some abnormalities of the respiratory tract.","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Migraines, epilepsy, and multiple sclerosis', 'Gastroesophageal reflux disease (GERD), irritable bowel syndrome (IBS), and peptic ulcers', 'Diabetes, hypertension, and stroke', 'Tuberculosis, infections, allergic reactions such as hay fever and allergic asthma, lung cancer, emphysema, and respiratory distress syndrome are some abnormalities of the respiratory tract.', 'Arthritis, osteoporosis, and gout', 'Kidney stones, gallstones, and hernias', 'Myocardial infarction, angina, and heart failure', 'Anemia, leukemia, and lymphoma', 'Psoriasis, eczema, and acne']\",\"question\":\"Briefly discuss some abnormalities of the respiratory tract.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}