{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ed1ff37-b8ae-5166-a1c5-5a458d02c5d6","task_key":"default--test--3988","task_revision_id":"1","upstream_id":"3988","short_description":"If 1.0 g of rubidium and 1.0 g of bromine are reacted, what will be left in…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['RbBr only', 'RbBr, Rb, Br2, and Rb2Br', 'RbBr and Rb2Br only', 'RbBr, Rb, and Br2', 'Rb and Br2 only', 'Br2 only', 'RbBr and Rb only', 'Rb only', 'Nothing will be left in measurable amounts', 'RbBr and Br2 only']\",\"question\":\"If 1.0 g of rubidium and 1.0 g of bromine are reacted, what will be left in measurable amounts (more than 0.10 mg) in the reaction vessel?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}