{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"783b52d8-f041-53ab-8faf-b17fb7fe24a2","task_key":"default--test--3811","task_revision_id":"1","upstream_id":"3811","short_description":"Which one of the following statements is true:","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Protons and neutrons possess orbital angular momentum only.', 'Protons and neutrons possess spin angular momentum only.', 'Protons possess orbital angular momentum only, neutrons possess spin angular momentum only.', 'Protons and neutrons have orbital and spin angular momentum.', 'Neutrons and protons possess both orbital and spin angular momentum.', 'Neutrons possess orbital angular momentum only, protons possess both orbital and spin angular momentum.', 'Protons and neutrons possess neither orbital nor spin angular momentum.', 'Neutrons have orbital and spin angular momentum, protons have spin angular momentum only.', 'Protons possess spin angular momentum only, neutrons possess orbital angular momentum only.', 'Protons have orbital and spin angular momentum, neutrons have spin angular momentum.']\",\"question\":\"Which one of the following statements is true:\",\"src\":\"ori_mmlu-college_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}