{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02ac9810-ccaf-54e8-aebd-c4072caccff9","task_key":"default--test--3269","task_revision_id":"1","upstream_id":"3269","short_description":"Hemoglobin is a molecule that binds to both O2 and CO2. There is an allosteric…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['increases as H+ concentration increases', 'decreases as blood pH decreases', 'decreases in resting muscle tissue', 'increases as blood pH decreases', 'increases as O2 concentration decreases', 'decreases as CO2 concentration decreases', 'remains constant regardless of O2 and CO2 concentrations', 'increases in exercising muscle tissue', 'increases as CO2 concentration decreases']\",\"question\":\"Hemoglobin is a molecule that binds to both O2 and CO2. There is an allosteric relationship between the concentrations of O2 and CO2. Hemoglobin's affinity for O2\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}