{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8501af79-e0d8-5040-a524-d81170c508af","task_key":"default--test--9483","task_revision_id":"1","upstream_id":"9483","short_description":"According to the BCS theory, the attraction between Cooper pairs in a…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Magnetic attraction', 'vacuum polarization', 'interactions with the ionic lattice', 'Gravitational pull', 'Electrostatic attraction', 'the weak nuclear force', 'The Pauli exclusion principle', 'Quantum entanglement', 'Interactions with the electron cloud', 'the strong nuclear force']\",\"question\":\"According to the BCS theory, the attraction between Cooper pairs in a superconductor is due to\",\"src\":\"ori_mmlu-college_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}