{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7f1de5e-743d-58f9-891b-21057e42f746","task_key":"default--test--4232","task_revision_id":"1","upstream_id":"4232","short_description":"Two flasks of equal volume are filled with nitrogen gas and, when both are…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['0.25 atm', '0.423 atm', '0.600 atm', '0.373 atm', '0.5 atm', '0.280 atm', '0.323 atm', '0.450 atm', '0.523 atm', '0.75 atm']\",\"question\":\"Two flasks of equal volume are filled with nitrogen gas and, when both are immersed in boiling water, the gas pressure inside the system is 0.5 atm. One of the flasks is then immersed in an ice-water mixture, keeping the other in the boiling water. Calculate the new pressure for the system.\",\"src\":\"stemez-PhysicalChemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}