{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4eb091a7-897b-505c-a3d9-bc37be4e256c","task_key":"default--test--3960","task_revision_id":"1","upstream_id":"3960","short_description":"What wavelength of light is needed to excite an electron in a 0.2 nanometer (1…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['14.510 nm, 24.412 nm', '19.510 nm, 29.412 nm', '15.510 nm, 25.412 nm', '20.510 nm, 30.412 nm', '22.510 nm, 32.412 nm', '13.510 nm, 23.412 nm', '18.510 nm, 28.412 nm', '21.510 nm, 31.412 nm', '16.510 nm, 26.412 nm', '17.510 nm, 27.412 nm']\",\"question\":\"What wavelength of light is needed to excite an electron in a 0.2 nanometer (1 nm = 10^-9 m) box from the ground state to the second excited state? What wavelength of light is emitted when the same electron falls from the second excited state to the first excited state?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}