{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67a17cab-ef8e-5669-bc66-409193285e10","task_key":"all--test--67a17cab-ef8e-5669-bc66-409193285e10","task_revision_id":"1","upstream_id":"","short_description":"Most transition metals share a common oxidation state of +2. Which of the…","config":"all","split":"test","body":"{\"choices\":\"['Transition metals all have a minimum of two unpaired electrons.', 'Transition metals have unstable configurations and are very reactive.', 'Transition metals tend to gain electrons when reacting with other elements.', 'Transition metals will lose their outermost s-block electrons when forming bonds.']\",\"question\":\"Most transition metals share a common oxidation state of +2. Which of the following best explains why?\",\"subject\":\"high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}