{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"21ee423a-f3ec-5d80-9cce-654e8c11d4c4","task_key":"default--test--6551","task_revision_id":"1","upstream_id":"6551","short_description":"A 49-year-old man, who is recovering in the hospital 2 days after uncomplicated…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Initiate antibiotic therapy', 'Computed Tomography (CT) of the lower extremity', 'Intra-arterial tissue plasminogen activator (tPA) therapy', 'Bedside compartment pressure measurements', 'Magnetic Resonance Imaging (MRI) of the left lower extremity', 'Doppler ultrasonography of the left lower extremity', 'Immediate surgical exploration', 'Pain management and observation', 'Intraoperative angiography', 'Intravenous heparin therapy']\",\"question\":\"A 49-year-old man, who is recovering in the hospital 2 days after uncomplicated left femoral-popliteal bypass grafting for claudication, has now developed increasing pain in his left foot. Until now, the patient's postoperative course had been unremarkable and he has been treated with low-dose morphine for pain control. Medical history is remarkable for type 2 diabetes mellitus controlled with metformin and diet. Vital signs now are temperature 36.8°C (98.2°F), pulse 80/min and regular, respirations 20/min, and blood pressure 150/92 mm Hg. The surgical incision appears clean and well approximated without abnormal erythema or swelling. The left lower extremity and foot appear pale. Palpation of the left lower extremity discloses a strong femoral pulse, a weak popliteal pulse, and a cool, pulseless foot. Which of the following is the most appropriate management? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}