{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb88f0ba-d54f-5a56-917a-95fb37cba2c3","task_key":"default--test--6034","task_revision_id":"1","upstream_id":"6034","short_description":"A 23-year-old woman with bone marrow failure is treated with a large dose of…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Activation of complement system', 'Overproduction of monocytes', 'Cytokine secretion by natural killer cells', 'Hyperactive B-lymphocyte response', 'Allergic reaction to rabbit proteins', 'Immune complex deposition in tissues', 'Polyclonal T-lymphocyte activation', 'Eosinophil degranulation', 'Increased production of interferon-gamma', 'Acute graft-versus-host disease']\",\"question\":\"A 23-year-old woman with bone marrow failure is treated with a large dose of rabbit antithymocyte globulin. Ten days later, she develops fever, lymphadenopathy, arthralgias, and erythema on her hands and feet. Which of the following is the most likely cause of these symptoms?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}