{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c081d1dd-9aeb-5374-92ff-957afdf78c5a","task_key":"default--test--3952","task_revision_id":"1","upstream_id":"3952","short_description":"A chemist reacts ferric sulfate with barium chloride and obtains barium sulfate…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['20.62 grams, 67.11 grams, 50.56 grams', '25.62 grams, 77.11 grams, 60.56 grams', '22.47 grams, 80.34 grams, 56.23 grams', '17.24 grams, 61.88 grams, 43.19 grams', '12.31 grams, 44.22 grams, 33.17 grams', '18.50 grams, 65.50 grams, 45.75 grams', '19.76 grams, 70.29 grams, 49.65 grams', '15.62 grams, 57.11 grams, 40.56 grams', '14.85 grams, 53.07 grams, 37.49 grams', '10.62 grams, 47.11 grams, 30.56 grams']\",\"question\":\"A chemist reacts ferric sulfate with barium chloride and obtains barium sulfate and ferric chloride. He writes the following balanced equation to express this reaction: Fe_2 (SO_4)_3 + 3BaCl_2 \\\\rightarrow 3BaSO_4\\\\downarrow + 2FeCl_3 (A)How much BaC1_2 should be used to react with 10 grams of Fe_2 (SO_4)_3? (B) How much Fe_2 (SO_4)_3 will be necessary to produce 100 g of BaSO_4? (C) From a mixture of 50 g of Fe_2 (SO_4)3 and 100 g of BaC1_2, how much FeC1_3 can be produced?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}