{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e24acfd3-b5d2-54e3-b1d5-db985de393bf","task_key":"default--test--6250","task_revision_id":"1","upstream_id":"6250","short_description":"A 16-year-old girl is brought to the office by her mother because she is…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Administer a tetanus shot', 'Prescribe azithromycin', 'Administer the meningococcal vaccine', 'Prescribe ibuprofen', 'Prescribe penicillin', 'Administer the pneumococcal vaccine', 'Prescribe amoxicillin', 'Prescribe doxycycline', 'Prescribe rifampin', 'No intervention is needed at this time']\",\"question\":\"A 16-year-old girl is brought to the office by her mother because she is concerned that she may have contracted meningitis from her soccer teammate, who was diagnosed with meningococcal meningitis and admitted to the hospital yesterday. The patient's soccer team traveled to a neighboring state to participate in a tournament 1 week ago and she shared a hotel room with the girl who was hospitalized. The patient feels well but is concerned she may have \\\"caught the same bug.\\\" Medical history is remarkable for asthma. Medications include inhaled albuterol. Vital signs are temperature 37.2°C (98.9°F), pulse 64/min, respirations 16/min, and blood pressure 107/58 mm Hg. Physical examination shows no abnormalities. Which of the following is the most appropriate intervention for this patient at this time?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}