{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3631e06e-d5ea-53dc-ad32-9851c417a344","task_key":"default--test--2295","task_revision_id":"1","upstream_id":"2295","short_description":"Functionally, receptors in the retina of the eye differ most from receptors in…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['magnitude of the resting potentials of their membranes', 'size and structure of their cells', 'ability to regenerate after damage', 'speed at which they transmit signals', 'number of axons each cell possesses', 'ions involved in their action potentials', 'degree of sensitivity to external stimuli', 'length of their refractory periods', 'location of their synapses', 'types of energy they transduce']\",\"question\":\"Functionally, receptors in the retina of the eye differ most from receptors in the cochlea of the ear in the\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}