{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2774702-dc08-586e-b917-0c6061751b8a","task_key":"default--test--4504","task_revision_id":"1","upstream_id":"4504","short_description":"Hund's rule requires that","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['all electrons must have opposite spins in an orbital', 'no two electrons can have the same four quantum numbers', 'no two electrons can occupy the same orbital', 'no two electrons with the same spin can occupy an orbital', 'all electrons must occupy the highest energy level first', 'no two electrons can have opposite spins in an orbital', 'no two electrons can occupy separate orbitals', 'all electrons must fill the lowest energy level first', 'no two electrons can pair up if there is an empty orbital at the same energy level available', 'no three electrons can occupy the same orbital']\",\"question\":\"Hund's rule requires that\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}