{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a11e44f4-3a04-5199-a756-64c0135186db","task_key":"default--test--6044","task_revision_id":"1","upstream_id":"6044","short_description":"A 16-year-old boy is admitted to the emergency department because of a knife…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Pneumothorax, not under tension', 'Flail chest', 'Pleural effusion', 'Pulmonary contusion', 'Tension pneumothorax', 'Hemothorax, not under tension', 'Hemopneumothorax, not under tension', 'Cardiac tamponade', 'Tension hemopneumothorax', 'Rib fracture']\",\"question\":\"A 16-year-old boy is admitted to the emergency department because of a knife wound to the left side of his chest. An x-ray of the chest shows an air-fluid level in the left side of the chest, partial collapse of the left lung, and elevation of the stomach bubble. The mediastinum is in the midline. Which of the following is the most likely diagnosis? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}