{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9c26dd1-a2b9-5c73-9c10-48a018365a55","task_key":"all--test--b9c26dd1-a2b9-5c73-9c10-48a018365a55","task_revision_id":"1","upstream_id":"","short_description":"A species of goose nests on both cliffs and beaches near the ocean. Soon after…","config":"all","split":"test","body":"{\"choices\":\"['Many more geese nest on the beaches than on the cliffs.', 'Cliff-side nesting confers a higher fitness than does beach nesting.', 'Chicks from cliff nests instinctively step off the cliffs at the appropriate time.', 'More chicks survive the fall from the cliffs than are killed.']\",\"question\":\"A species of goose nests on both cliffs and beaches near the ocean. Soon after hatching, all chicks must make their way to the ocean. Chicks from cliff nests must tumble down the cliff to get to the ocean, and many are killed by the fall. Which of the following is most consistent with the hypothesis that cliff nesting is adaptive in this goose species?\",\"subject\":\"college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}