{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e69df337-75e1-53b9-8f4d-cd6327831d0e","task_key":"default--test--2653","task_revision_id":"1","upstream_id":"2653","short_description":"A 7-year-old student arrived in the United States one year ago from a…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"The discrepancy between the student's native language reading skills and English reading skills is related to the greater complexity of English.\\\", \\\"The discrepancy between the student's native language reading skills and English reading skills is due to the different teaching methodologies in her previous and current schools.\\\", \\\"The student's high scores in reading in her native language are misleading her actual reading capacity in English.\\\", \\\"The student's reading difficulty is due to her inability to adjust to the new learning environment.\\\", \\\"The student's difficulty in reading is due to lack of exposure to English literature.\\\", \\\"The student's reading skills are being hindered due to her proficiency in speaking and listening.\\\", \\\"The discrepancy between the student's English social language skills and reading skills is expected given the richer context in which social skills are acquired.\\\", \\\"The student's reading difficulty is due to lack of interest in the subject matter in the basal reader.\\\", \\\"The student's reading difficulty is an early indicator that she will have increased academic problems as her coursework requires more reading.\\\", \\\"Continued use of the student's native language in her home environment is interfering with her development of English.\\\"]\",\"question\":\"A 7-year-old student arrived in the United States one year ago from a non-Englishspeaking country, where she achieved high scores in reading. Over the year, she has become fluent in social English. After a few months in a monolingual English second grade, her teacher refers her for evaluation because she has great difficulty with the basal reader used in the class. Two English proficiency tests administered to the student show that she performs above the mean for monolingual English grade peers in speaking and listening but well below the mean in reading and writing. She also performs well above the mean for grade peers on reading tests in her native language. Based on this information alone, which of the following is the most accurate interpretation?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}