{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"436bfa26-84a7-53b2-a05d-ee954b2c5be0","task_key":"default--test--4628","task_revision_id":"1","upstream_id":"4628","short_description":"A chemist discovered a new compound and wished to determine its molecular…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['400 g/mole', '250 g/mole', '305 g/mole', '500 g/mole', '325 g/mole', '275 g/mole', '450 g/mole', '350 g/mole', '150 g/mole', '180 g/mole']\",\"question\":\"A chemist discovered a new compound and wished to determine its molecular weight. He placed 2.00 grams of the unknown compound in 10 grams of water and found that the normal freezing point of the water was depressed to-1.22°C. What is the molecular weight of the new compound?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}