{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6414a9ae-1576-54ea-bc6e-15b55b209cd8","task_key":"default--test--4077","task_revision_id":"1","upstream_id":"4077","short_description":"An experiment was performed to determine the moles of carbon dioxide gas formed…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Perhaps there was too much acid, which interfered with the results.', 'The measurement of atmospheric pressure was incorrect.', 'Carbon dioxide is rather soluble in water.', 'The temperature might have affected the reaction.', 'Limestone is Ca(HCO3)2.', 'The limestone did not fully disappear.', 'Limestone is never pure CaCO3.', 'Perhaps there was not enough acid to dissolve all the limestone.', 'The volume of CO2 was not measured correctly.', 'The vapor pressure of water was not correctly accounted for.']\",\"question\":\"An experiment was performed to determine the moles of carbon dioxide gas formed (collected over water) when an acid reacts with limestone. To do this, a piece of dry limestone was weighed. Then carbon dioxide was collected until the limestone disappeared. The atmospheric pressure was measured at 0.965 atm. The temperature was 295 K. The volume of CO2 was measured to the nearest mL and corrected for the vapor pressure of water. The student continuously got low results from what was expected. Can you suggest why?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}