{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ba839c7-fdf6-5f4d-b5c6-084708e5cb14","task_key":"all--test--4ba839c7-fdf6-5f4d-b5c6-084708e5cb14","task_revision_id":"1","upstream_id":"","short_description":"A healthy person sitting at rest consciously begins to breathe too fast. Which…","config":"all","split":"test","body":"{\"choices\":\"['A decrease in partial pressure of carbon dioxide and an increase in concentration of hydrogen ions', 'A decrease in partial pressure of carbon dioxide and a decrease in concentration of hydrogen ions', 'An increase in partial pressure of carbon dioxide and an increase in concentration of hydrogen ions', 'An increase in partial pressure of carbon dioxide and a decrease in concentration of hydrogen ions']\",\"question\":\"A healthy person sitting at rest consciously begins to breathe too fast. Which of the following describes the change in the person’s blood caused by this hyperventilation?\",\"subject\":\"college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}