{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4af1a72c-1b6e-51da-951c-752749c67b04","task_key":"default--test--12056","task_revision_id":"1","upstream_id":"12056","short_description":"What is the equilibrium composition of a mixture ofcarbon monoxide, carbon…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['x_CO= 0.099, x_(_O)2 = 0.60, x_C_(_O)2 = 0.30', 'x_CO= 0.081, x_(_O)2 = 0.65, x_C_(_O)2 = 0.26', 'x_CO= 0.087, x_(_O)2 = 0.62, x_C_(_O)2 = 0.293', 'x_CO= 0.0891, x_(_O)2 = 0.615, x_C_(_O)2 = 0.296', 'x_CO= 0.075, x_(_O)2 = 0.675, x_C_(_O)2 = 0.25', 'x_CO= 0.093, x_(_O)2 = 0.59, x_C_(_O)2 = 0.317', 'x_CO= 0.095, x_(_O)2 = 0.605, x_C_(_O)2 = 0.3', 'x_CO= 0.091, x_(_O)2 = 0.61, x_C_(_O)2 = 0.29', 'x_CO= 0.082, x_(_O)2 = 0.63, x_C_(_O)2 = 0.287', 'x_CO= 0.085, x_(_O)2 = 0.58, x_C_(_O)2 = 0.335']\",\"question\":\"What is the equilibrium composition of a mixture ofcarbon monoxide, carbon dioxide, and molecular oxygen at 3000°K and30psia? Assume the mixture contains 5 atoms of oxygenper atom of carbon.\",\"src\":\"stemez-Thermodynamics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}