{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da40115c-4f36-58b3-a1aa-dc4947a19ee7","task_key":"default--test--11445","task_revision_id":"1","upstream_id":"11445","short_description":"Ten grams of argon gas at an initial pressure and temperatureof 608kPa, 300°K…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['Final pressure is 202.7kPa, Final temperature is 300K, Entropy change is +2.286 J/K', 'Final pressure is 152kPa, Final temperature is 450K, Entropy change is +1.714 J/K', 'Final pressure is 202.7kPa, Final temperature is 200K, Entropy change is +0.5715 J/K', 'Final pressure is 202.7kPa, Final temperature is 900K, Entropy change is +3.429 J/K', 'Final pressure is 608kPa, Final temperature is 300K, Entropy change is +0.2286 J/K', 'Final pressure is 300kPa, Final temperature is 300K, Entropy change is -0.2286 J/K', 'Final pressure is 300kPa, Final temperature is 202.7K, Entropy change is +2.286 J/K', 'Final pressure is 202.7kPa, Final temperature is 100K, Entropy change is +1.143 J/K', 'Final pressure is 202.7kPa, Final temperature is 600K, Entropy change is -2.286 J/K', 'Final pressure is 404.3kPa, Final temperature is 300K, Entropy change is +1.143 J/K']\",\"question\":\"Ten grams of argon gas at an initial pressure and temperatureof 608kPa, 300°K respectively undergo a changeof state at constant internal energy until the final volumeis three times the initial occupied volume. Assuming idealgas behavior determine the final state (pressure and temperature), and the entropy change of the gas due to the changeof state.\",\"src\":\"stemez-Thermodynamics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}