{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c19b05f8-44df-5824-94d5-0ba40cc4fcac","task_key":"all--test--c19b05f8-44df-5824-94d5-0ba40cc4fcac","task_revision_id":"1","upstream_id":"","short_description":"A piano teacher is helping a student learn a new piece. At first, she praises…","config":"all","split":"test","body":"{\"choices\":\"['Negative reinforcement', 'Negative punishment', 'Shaping', 'Chaining']\",\"question\":\"A piano teacher is helping a student learn a new piece. At first, she praises the student for playing correct notes. After the student begins to show proficiency with the notes, the teacher only praises the student when he adds the proper dynamics and interpretation to the segments he plays. The teacher's method of instruction is an example of which of the following types of training techniques?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}