{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e2d630d-8d4a-5c7d-bc34-324bdbc81d1d","task_key":"default--test--4441","task_revision_id":"1","upstream_id":"4441","short_description":"In 1976 it was mistakenly believed that the first of the 'superheavy' elements…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['0.12 $\\\\\\\\mathrm{pm}$', '0.72 $\\\\\\\\mathrm{pm}$', '0.82 $\\\\\\\\mathrm{pm}$', '0.22 $\\\\\\\\mathrm{pm}$', '0.02 $\\\\\\\\mathrm{pm}$', '0.62 $\\\\\\\\mathrm{pm}$', '0.42 $\\\\\\\\mathrm{pm}$', '0.32 $\\\\\\\\mathrm{pm}$', '0.52 $\\\\\\\\mathrm{pm}$', '0.92 $\\\\\\\\mathrm{pm}$']\",\"question\":\"In 1976 it was mistakenly believed that the first of the 'superheavy' elements had been discovered in a sample of mica. Its atomic number  was believed to be 126. What is the most probable distance of the innermost electrons from the nucleus of an atom of this element? (In such elements, relativistic effects are very important, but ignore them here.)\",\"src\":\"scibench-matter\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}