{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28fbdf41-f04e-5286-9c87-6782a6bac1b6","task_key":"default--test--6223","task_revision_id":"1","upstream_id":"6223","short_description":"Which of the following statements is true of the pupillary light reflex?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Its afferent limb is carried in the oculomotor nerve', 'It is mediated by the inferior colliculi in the midbrain', 'It is a unilateral reflex', 'It is mediated by the lateral geniculate body in the thalamus', 'Its afferent limb is carried in the trigeminal nerve', 'Its efferent limb is carried in the optic nerve', 'Its efferent limb is carried in the oculomotor nerve', 'Its afferent limb is carried in the optic nerve', 'It is mediated by the superior colliculi in the midbrain', 'It is a consensual reflex']\",\"question\":\"Which of the following statements is true of the pupillary light reflex?\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}