{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"794091a1-b755-5a20-8129-5f7bd983e50a","task_key":"default--test--3724","task_revision_id":"1","upstream_id":"3724","short_description":"Most transition metals share a common oxidation state of +2. Which of the…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Transition metals will gain their outermost s-block electrons when forming bonds.', 'Transition metals tend to gain electrons when reacting with other elements.', 'Transition metals tend to lose electrons when reacting with other elements.', 'Transition metals all have a minimum of two unpaired electrons.', 'Transition metals have a maximum of two unpaired electrons.', 'Transition metals will lose their outermost s-block electrons when forming bonds.', 'Transition metals will share their outermost s-block electrons when forming bonds.', 'Transition metals have a +2 charge in their ground state.', 'Transition metals have a stable configuration and are not reactive.', 'Transition metals have unstable configurations and are very reactive.']\",\"question\":\"Most transition metals share a common oxidation state of +2. Which of the following best explains why?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}