{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d436fdbe-f7c1-5693-be0b-8c2c51c4e7ba","task_key":"default--test--3629","task_revision_id":"1","upstream_id":"3629","short_description":"The Pauli exclusion principle states that","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['no two electrons can occupy separate orbitals', 'no two protons can have the same four quantum numbers', 'two electrons can share the same quantum number if they are in different orbitals', 'no two electrons can have the same four quantum numbers', 'no two electrons can pair up if there is an empty orbital available', 'no two electrons with the same spin can occupy an orbital', 'two electrons can share the same quantum number if they have opposite spins', 'all electrons in an atom must have the same spin']\",\"question\":\"The Pauli exclusion principle states that\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}