{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec9f9862-2037-597d-96bc-505ffd14a62b","task_key":"default--test--2995","task_revision_id":"1","upstream_id":"2995","short_description":"A healthy person sitting at rest consciously begins to breathe too fast. Which…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['No change in both partial pressure of carbon dioxide and concentration of hydrogen ions', 'A decrease in partial pressure of carbon dioxide and a decrease in concentration of hydrogen ions', 'No change in partial pressure of carbon dioxide and an increase in concentration of hydrogen ions', 'A decrease in partial pressure of carbon dioxide and an increase in concentration of hydrogen ions', 'An increase in partial pressure of carbon dioxide and a decrease in concentration of hydrogen ions', 'A decrease in partial pressure of carbon dioxide and no change in concentration of hydrogen ions', 'An increase in both partial pressure of carbon dioxide and concentration of hydrogen ions, but at different rates', 'An increase in partial pressure of carbon dioxide and no change in concentration of hydrogen ions', 'No change in partial pressure of carbon dioxide and a decrease in concentration of hydrogen ions', 'An increase in partial pressure of carbon dioxide and an increase in concentration of hydrogen ions']\",\"question\":\"A healthy person sitting at rest consciously begins to breathe too fast. Which of the following describes the change in the person’s blood caused by this hyperventilation?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}