{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0839a39d-d19e-512a-acfa-514214e58831","task_key":"all--test--0839a39d-d19e-512a-acfa-514214e58831","task_revision_id":"1","upstream_id":"","short_description":"After a visit to a rescue squad, Ms. Espinosa asks her 3-year-old students to…","config":"all","split":"test","body":"{\"choices\":\"['Paraphrasing ideas', 'Spelling words correctly', 'Expressing speech in print', 'Capitalizing proper nouns']\",\"question\":\"After a visit to a rescue squad, Ms. Espinosa asks her 3-year-old students to help her write a thank-you letter to the rescue squad employees. She records the students' responses on a large sheet of chart paper as the students share their ideas. While writing, Ms. Espinosa focuses the students' attention on several early literacy skills. Which of the following is the most appropriate focus for 3-year-old students?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}