{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"379c13cd-314c-56fd-9b26-d521b0341bb1","task_key":"default--test--6043","task_revision_id":"1","upstream_id":"6043","short_description":"A 16-year-old boy is admitted to the emergency department because of a knife…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Rib fracture', 'Hemopneumothorax, not under tension', 'Flail chest', 'Pneumothorax, not under tension', 'Cardiac tamponade', 'Tension pneumothorax', 'Pulmonary contusion', 'Tension hemopneumothorax', 'Hemothorax, not under tension', 'Pleural effusion']\",\"question\":\"A 16-year-old boy is admitted to the emergency department because of a knife wound to the left side of his chest. An x-ray of the chest shows an air-fluid level in the left side of the chest, partial collapse of the left lung, and elevation of the stomach bubble. The mediastinum is in the midline. Which of the following is the most likely diagnosis?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}