{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f7b1ff5-4789-56cd-aa8d-a7c3c51c2047","task_key":"all--test--2f7b1ff5-4789-56cd-aa8d-a7c3c51c2047","task_revision_id":"1","upstream_id":"","short_description":"Grammar may be taught in two main ways‚Äî by experience with discourse that…","config":"all","split":"test","body":"{\"choices\":\"['using language in a wide variety of situations improves grammar', 'good judgment can be improved by studying the rules of formal grammar', 'analyzing and diagramming provide exercise in logical thinking', 'formal study of grammar improves writing ability']\",\"question\":\"Grammar may be taught in two main ways‚Äî by experience with discourse that entails the varieties of word forms and sentence construction, or by analyzing dummy sentences and diagramming parts. Plentiful discursive experience is what really teaches grammar, for it exercises judgment and provides language intake, whereas formal grammar study has been proved irrelevant. Politics more than pedagogy retards the changing of the curriculum to fit this truth. The author of the passage above argues that\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}