{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4997472-ef07-5fd8-aa23-f59d45fe8ea2","task_key":"default--test--9090","task_revision_id":"1","upstream_id":"9090","short_description":"What is the energy of a photon of blue light whose frequency is7 × 101 4Hz and…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['1.2 × 10^-20 J, 3.3 × 10^-27 J', '5.8 × 10^-18 J, 7.2 × 10^-25 J', '3.6 × 10^-18 J, 7.7 × 10^-25 J', '6.2 × 10^-18 J, 4.4 × 10^-25 J', '4.6 × 10^-19 J, 6.6 × 10^-26 J', '7.5 × 10^-19 J, 5.0 × 10^-26 J', '5.0 × 10^-19 J, 2.2 × 10^-26 J', '4.1 × 10^-19 J, 1.1 × 10^-25 J', '2.9 × 10^-19 J, 9.9 × 10^-26 J', '3.4 × 10^-19 J, 8.8 × 10^-26 J']\",\"question\":\"What is the energy of a photon of blue light whose frequency is7 × 101 4Hz and the energy of a photon for FM electromagneticradiation if the frequency is 1 × 10^8 Hz?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}