{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13320f42-5097-56be-95d7-53f5fec6e663","task_key":"default--test--9380","task_revision_id":"1","upstream_id":"9380","short_description":"The half-cell potential for the reaction $\\mathrm{O}_2(g)+4 \\mathrm{H}^{+}(a…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['4.16 $10^{-4}$', '8.50 $10^{-4}$', '1.09 $10^{-4}$', '3.00 $10^{-4}$', '1.50 $10^{-4}$', '5.78 $10^{-4}$', '2.32 $10^{-4}$', '9.75 $10^{-4}$', '6.25 $10^{-4}$', '7.00 $10^{-4}$']\",\"question\":\"The half-cell potential for the reaction $\\\\mathrm{O}_2(g)+4 \\\\mathrm{H}^{+}(a q)+4 \\\\mathrm{e}^{-} \\\\rightarrow 2 \\\\mathrm{H}_2 \\\\mathrm{O}(l)$ is $+1.03 \\\\mathrm{~V}$ at $298.15 \\\\mathrm{~K}$ when $a_{\\\\mathrm{O}_2}=1.00$. Determine $a_{\\\\mathrm{H}^{+}}$\",\"src\":\"scibench-thermo\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}