{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94f03cd1-b7ae-5794-8d0e-184f3cf4e00b","task_key":"default--test--6492","task_revision_id":"1","upstream_id":"6492","short_description":"Three weeks ago a 45-year-old man was admitted to the hospital because of…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Hyperbaric oxygen', 'Skin grafting', 'Intravenous fluids and electrolytes', 'Application of topical collagenase', 'Application of silver sulfadiazine', 'Amputation', 'Application of warm compresses', 'Placement of a wound VAC', 'Debridement of necrotic skin over the toes']\",\"question\":\"Three weeks ago a 45-year-old man was admitted to the hospital because of frostbite of both feet. He was treated by rapid rewarming and protective care of the feet. All the toes on the right foot have turned black. He has become slightly febrile and progressively more confused during the past few days. Examination discloses cellulitis in the midfoot. Which of the following is the most appropriate treatment?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}