{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9aa9dd65-527b-547d-b578-d046007bc9e7","task_key":"all--test--9aa9dd65-527b-547d-b578-d046007bc9e7","task_revision_id":"1","upstream_id":"","short_description":"During cooperative learning, all of the students in Group A members initially…","config":"all","split":"test","body":"{\"choices\":\"['both groups would moderate their positions', 'Group A members would moderate their positions, but Group B members would retain their original positions', 'Group A would become more firmly entrenched, while Group B members would moderate their positions', 'Group A members would retain their original positions, but Group B members would moderate their positions']\",\"question\":\"During cooperative learning, all of the students in Group A members initially were opposed to the death penalty, whereas two of the students in Group B members were opposed to the death penalty and two were in favor of the death penalty. According to research, after an intense discussion within each group about capital punishment, we would expect\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}