{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e3085b66-c23a-5bf1-8744-73ad000fe760","task_key":"default--test--2485","task_revision_id":"1","upstream_id":"2485","short_description":"What are the vestibular organs, and how do the operate?","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The vestibular organs are the semicircular canals and theotolithorgans. They provide us with a sense of balance through information about the movements and position of our head.', 'The vestibular organs are the malleus and incus, which are involved in the amplification of sound within the ear.', 'The vestibular organs are the hair cells and basilar membrane, which are critical for the perception of pitch and volume in hearing.', 'The vestibular organs are the pinna and earlobe', 'The vestibular organs are the auditory canal and eardrum', 'The vestibular organs are the cochlea and auditory nerves', 'The vestibular organs are the stapes and round window, which play roles in the conversion of sound waves into mechanical energy.', 'The vestibular organs are the tympanic membrane and stapedius muscle, which contribute to the detection of sound frequency.', 'The vestibular organs are the Eustachian tube and mastoid process, which are related to the regulation of air pressure in the middle ear.', 'The vestibular organs are the vestibular nerve and cochlear duct, which are responsible for transmitting sound vibrations to the brain.']\",\"question\":\"What are the vestibular organs, and how do the operate?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}