{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b161aa7f-4006-51f4-9536-1c97284f08ea","task_key":"all--test--b161aa7f-4006-51f4-9536-1c97284f08ea","task_revision_id":"1","upstream_id":"","short_description":"A quote from a natural resources text states: \"Whenever the original ecosystem…","config":"all","split":"test","body":"{\"choices\":\"['there exist large populations of a low number of species', 'population levels of a species are kept at equilibrium through natural regulatory mechanisms', 'a given prey organism is subject to higher predation rates by more diverse predators', 'a given prey organism is less likely to undergo a population surge']\",\"question\":\"A quote from a natural resources text states: \\\"Whenever the original ecosystem becomes restructured by man, it tends to become simplified, with a resultant disruption of the stabilizing influences of density-dependent regulatory factors.\\\" This implies that in a disturbed ecosystem\",\"subject\":\"college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}