{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49f61229-94c9-5700-a752-7729af0c163b","task_key":"default--test--4378","task_revision_id":"1","upstream_id":"4378","short_description":"You have two 1-liter containers connected to each other by a valve which is…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Vapor pressure is halved, the concentration of the gaseous molecules is doubled, and the number of molecules is doubled', 'Vapor pressure is unchanged, the concentration of the gaseous molecules is halved, and the number of molecules is doubled', 'Vapor pressure, the concentration of the gaseous molecules and the number of molecules are all unchanged', 'Vapor pressure is doubled, the concentration of the gaseous molecules is doubled, and the number of molecules is halved', 'Vapor pressure is doubled, the concentration of the gaseous molecules is halved, and the number of molecules is unchanged', 'Vapor pressure and the concentration of the gaseous molecules are halved, the number of molecules remains unchanged', 'Vapor pressure is halved, the concentration of the gaseous molecules remains unchanged, and the number of molecules is doubled', 'Vapor pressure and the concentration of the gaseous molecules remain unchanged, the number of molecules is doubled', 'Vapor pressure and the concentration of the gaseous molecules are doubled, the number of molecules remains unchanged', 'Vapor pressure is doubled, the concentration of the gaseous molecules and the number of molecules remain unchanged']\",\"question\":\"You have two 1-liter containers connected to each other by a valve which is closed. In one container, you have liquid water in equilibrium with water vapor at 25°C. The other container contains a vacuum. Suddenly, you open the valve. Discuss the changes that take place, assuming temperature is constant with regard to (a) the vapor pressure, (b) the concentration of the water molecules in the vapor, (c) the number of molecules in the vapor state.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}