{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f29c9c2-d299-54d9-a5f8-13623cbd05af","task_key":"default--test--4591","task_revision_id":"1","upstream_id":"4591","short_description":"Chlorine is found in nature in two isotopic forms, one of atomic mass 35amuand…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['90%, 10%', '65%, 35%', '77.35%, 22.65%', '50%, 50%', '60%, 40%', '75%, 25%', '82.35%, 17.65%', '70%, 30%', '80%, 20%', '85%, 15%']\",\"question\":\"Chlorine is found in nature in two isotopic forms, one of atomic mass 35amuand one of atomic mass 37amu. The average atomic mass of chlorine is 35.453amu. What is the percent with which each of these isotopes occurs in nature?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}