{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a63dd429-4904-5314-958e-4e7df56a0c46","task_key":"default--test--6493","task_revision_id":"1","upstream_id":"6493","short_description":"A 35-year-old man comes to the physician because of pain and swelling of his…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Disruption of vascular basement membranes', 'Decreased plasma protein concentration', 'Reduced capillary hydrostatic pressure', 'Increased capillary permeability due to inflammation', 'Reduced lymphatic drainage', 'Release of thromboxane', 'High blood pressure', 'Release of histamine', 'Separation of endothelial junctions', 'Increased hydrostatic pressure']\",\"question\":\"A 35-year-old man comes to the physician because of pain and swelling of his right arm where he scraped it on a tree branch 2 days ago. His temperature is 38.3°C (101°F). Examination of the right forearm shows edema around a fluctuant erythematous lesion at the site of trauma. The area is extremely tender to palpation. Which of the following is most likely the primary mechanism of the development of edema in this patient?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}