{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93a4e272-babe-5e18-97df-3c5736388892","task_key":"all--test--93a4e272-babe-5e18-97df-3c5736388892","task_revision_id":"1","upstream_id":"","short_description":"Genetic studies of the two primary varieties of rice, japonica (short-grained),…","config":"all","split":"test","body":"{\"choices\":\"['each is more closely related to one another than their local wild ancestors, suggesting that these modern varieties of rice were developed recently in east Asia.', 'each is more closely related to local wild ancestors than one another, suggesting that these modern varieties of rice were domesticated separately in east Asia and south Asia.', 'each is unrelated to one another, suggesting that these modern varieties of rice were domesticated independently in east Asia and western Europe.', 'each is closely related to their local wild ancestors, suggesting that domestication must have occurred very recently and no earlier than 2,000 years ago.']\",\"question\":\"Genetic studies of the two primary varieties of rice, japonica (short-grained), and indica (long-grained), suggest that:\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}