{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"def29371-9bfb-59c7-bb9f-47463c7bdb24","task_key":"all--test--def29371-9bfb-59c7-bb9f-47463c7bdb24","task_revision_id":"1","upstream_id":"","short_description":"As an experiment, a group of newborn kittens was allowed to see through only one…","config":"all","split":"test","body":"{\"choices\":\"['They were unable to make use of interposition depth perception.', 'They were unable to maintain perceptual constancy.', 'They were unable to distinguish left from right monocular cues.', 'They were unable to use binocular cues for depth perception.']\",\"question\":\"As an experiment, a group of newborn kittens was allowed to see through only one eye at a time. Each day one of the eyes would be covered, switching between the two eyes on subsequent days. Which of the following best describes the visual limitations experienced by these cats as adults?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}