{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"481bc620-5c77-5d59-9000-b49947b1864b","task_key":"all--test--481bc620-5c77-5d59-9000-b49947b1864b","task_revision_id":"1","upstream_id":"","short_description":"A kindergarten teacher has students determine the following. ‚Ä¢ How many…","config":"all","split":"test","body":"{\"choices\":\"['Classifying', 'Counting', 'Patterning', 'Ordering']\",\"question\":\"A kindergarten teacher has students determine the following. ‚Ä¢ How many students are absent that day ‚Ä¢ If there are enough snack cups for each student to have one during snack time ‚Ä¢ How many balls are taken outside during recess and how many are brought back inside after recess. Which of the following concepts is most closely aligned with the determinations the teacher is having the students make?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}