{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9ed25c3-10eb-578a-aac3-f3e69d7804ae","task_key":"default--test--3835","task_revision_id":"1","upstream_id":"3835","short_description":"Lunar soil contains,ilmenite, a mineral with the composition FeO\\bulletTiO_2. It…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['600 g H_2O and 100 g O_2', '350 g H_2O and 150 g O_2', '450 g H_2O and 400 g O_2', '150 g H_2O and 250 g O_2', '500 g H_2O and 200 g O_2', '296 g H_2O and 262 g O_2', '250 g H_2O and 550 g O_2', '100 g H_2O and 300 g O_2', '400 g H_2O and 350 g O_2', '200 g H_2O and 500 g O_2']\",\"question\":\"Lunar soil contains,ilmenite, a mineral with the composition FeO\\\\bulletTiO_2. It has been suggested that moon explorers might obtain water and oxygen from lunar soil by reducing it with hydrogen (brought from earth) according to the following reactions. Beginning with soil heated to 1300°C in a solar furnace: FeTiO_3 + H_2-----------------\\\\rightarrow H_2O + Fe + TiO_2 Electrolysis 2H_2O-----------------\\\\rightarrow 2H_2O + O_2 How much water and oxygen could be obtained from 50 kg of soil if the soil is taken to be 5 per centilmenite?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}