{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e5d24a4-d695-5c13-83d7-99af84e1c3dd","task_key":"default--test--3992","task_revision_id":"1","upstream_id":"3992","short_description":"J.G. Dojahn, et al. (J. Phys. Chem. 100, 9649 (1996)) characterized the…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['235.1 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '199.4 $\\\\\\\\mathrm{~J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '183.2 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '220.8 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '250.3 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '162.9 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '208.5 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '215.6 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '174.7 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$', '188.6 $\\\\\\\\mathrm{J} \\\\\\\\mathrm{~mol}^{-1} \\\\\\\\mathrm{~K}^{-1}$']\",\"question\":\"J.G. Dojahn, et al. (J. Phys. Chem. 100, 9649 (1996)) characterized the potential energy curves of the ground and electronic states of homonuclear diatomic halogen anions. The ground state of $\\\\mathrm{F}_2^{-}$is ${ }^2 \\\\sum_{\\\\mathrm{u}}^{+}$with a fundamental vibrational wavenumber of $450.0 \\\\mathrm{~cm}^{-1}$ and equilibrium internuclear distance of $190.0 \\\\mathrm{pm}$. The first two excited states are at 1.609 and $1.702 \\\\mathrm{eV}$ above the ground state. Compute the standard molar entropy of $\\\\mathrm{F}_2^{-}$at $298 \\\\mathrm{~K}$.\",\"src\":\"scibench-matter\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}