{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa3ade66-3158-5e9e-bd30-3c42980c16a4","task_key":"all--test--aa3ade66-3158-5e9e-bd30-3c42980c16a4","task_revision_id":"1","upstream_id":"","short_description":"A 7-year-old student arrived in the United States one year ago from a…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Continued use of the student's native language in her home environment is interfering with her development of English.\\\", \\\"The student's reading difficulty is an early indicator that she will have increased academic problems as her coursework requires more reading.\\\", \\\"The discrepancy between the student's English social language skills and reading skills is expected given the richer context in which social skills are acquired.\\\", \\\" The discrepancy between the student's native language reading skills and English reading skills is related to the greater complexity of English.\\\"]\",\"question\":\"A 7-year-old student arrived in the United States one year ago from a non-Englishspeaking country, where she achieved high scores in reading. Over the year, she has become fluent in social English. After a few months in a monolingual English second grade, her teacher refers her for evaluation because she has great difficulty with the basal reader used in the class. Two English proficiency tests administered to the student show that she performs above the mean for monolingual English grade peers in speaking and listening but well below the mean in reading and writing. She also performs well above the mean for grade peers on reading tests in her native language. Based on this information alone, which of the following is the most accurate interpretation?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}