{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"007ceacb-f644-5386-8e10-c707b7887dd2","task_key":"default--test--9352","task_revision_id":"1","upstream_id":"9352","short_description":"The sodium doublet has wavelength of 589.0 nm and 589.6 nm. If the diffraction…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['560 lines', '520 lines', '500 lines', '600 lines', '470 lines', '450 lines', '480 lines', '510 lines', '430 lines', '491 lines']\",\"question\":\"The sodium doublet has wavelength of 589.0 nm and 589.6 nm. If the diffraction grating is blazed so as to be used best in the second order, at least how many lines must the grating have in order to resolve the doublet?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}