{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06bde26f-be06-57c9-859d-3fd0bb7831ee","task_key":"default--test--6805","task_revision_id":"1","upstream_id":"6805","short_description":"A 22-year-old woman comes to the emergency department because of a 5-day history…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Oral nystatin therapy', 'Tranquilizer therapy', 'Intravenous methylprednisolone therapy', 'Oral pain reliever therapy', 'Intravenous antiviral therapy', 'Incision and drainage', 'Salt water gargle', 'Antacid therapy', 'Oral antifungal therapy', 'Inhaled corticosteroid therapy']\",\"question\":\"A 22-year-old woman comes to the emergency department because of a 5-day history of sore throat and fever. During the past 3 days, she also has noticed a change in her voice and has had increased pain with swallowing. She has been taking acetaminophen since her symptoms began. Medical history is unremarkable and she takes no routine medications. Vital signs are temperature 40.0°C (104.0°F), pulse 130/min, respirations 18/min, and blood pressure 102/66 mm Hg. Pulse oximetry on room air shows an oxygen saturation of 97%. Examination of the oropharynx shows erythema, edema, and anterior displacement of the right side of the soft palate. The uvula is deviated to the left. There is a white exudate on the right tonsil. Trismus is noted. Palpation of the neck discloses enlarged, tender, mobile anterior cervical lymph nodes. In addition to antibiotic therapy, which of the following is the most appropriate management?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}