{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c687ce15-eb12-5bc4-83b9-6b5f61e7b928","task_key":"default--test--3869","task_revision_id":"1","upstream_id":"3869","short_description":"\"Hard\" water contains small amounts of the salts calcium bicarbonate…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['8 tons', '35 tons', '25 tons', '12 tons', '50 tons', '30 tons', '15 tons', '20 tons', '5 tons', '10 tons']\",\"question\":\"\\\"Hard\\\" water contains small amounts of the salts calcium bicarbonate (Ca(HCO_3)_2) and calcium sulfate (CaSO_4, molecular weight = 136 g/mole). These react with soap before it has a chance to lather, which is responsible for its cleansing ability. Ca(HCO_3)_2 is removed by boiling to form insoluble CaCO_3. CaSO_4, is removed by reaction with washing soda (Na_2 CO_3, molecular weight = 106 g/mole) according to the following equation: CaSO_4 + Na_2 CO_3 \\\\rightarrow CaCO_3 + Na_2 SO_4. If the rivers surrounding New York City have a CaSO_4 concentration of 1.8 × 10^-3 g/liter, how much Na_2 CO_3 is required to \\\"soften\\\" (remove CaSO_4) the water consumed by the city in one day (about 6.8 × 10^9 liters)?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}