{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ae71e84-856d-51a4-849c-7aca2e42c93a","task_key":"default--test--6376","task_revision_id":"1","upstream_id":"6376","short_description":"A previously healthy 27-year-old nulligravid woman comes to the emergency…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Administration of intravenous ceftriaxone', 'Observation and re-evaluation in 24 hours', 'Initiation of antiviral therapy', 'Ultrasound-guided needle aspiration of the mass', 'Incision and drainage', 'Administration of oral antibiotics', 'Administration of intravenous vancomycin', 'Administration of intravenous penicillin G', 'Surgical excision of the mass', 'Administration of intravenous metronidazole']\",\"question\":\"A previously healthy 27-year-old nulligravid woman comes to the emergency department because of a 2-day history of moderate-to-severe pain and swelling of the left labia. She is sexually active and uses condoms inconsistently. Her temperature is 37.2°C (99°F), pulse is 92/min, respirations are 18/min, and blood pressure is 115/75 mm Hg. Pelvic examination shows a 4 x 3-cm, tender, fluctuant mass medial to the left labium majus compromising the introital opening. Which of the following is the most appropriate next step in management? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}