{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2da029d-034d-5b1a-949b-59cfdc83b456","task_key":"all--test--a2da029d-034d-5b1a-949b-59cfdc83b456","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is the strongest piece of evidence for the idea that…","config":"all","split":"test","body":"{\"choices\":\"['The discovery of physical structures in animal brains are analogous to the language acquisition device in humans.', 'Primates quickly learn words that result in food rewards.', 'Systems of communication have been documented in species from honeybees to dolphins.', 'Apes have been able to use the words they know to express novel concepts.']\",\"question\":\"Which of the following is the strongest piece of evidence for the idea that animals are capable of developing and using a sophisticated language?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}