{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e541ef58-2129-5180-9079-bbddfda0e80a","task_key":"all--test--e541ef58-2129-5180-9079-bbddfda0e80a","task_revision_id":"1","upstream_id":"","short_description":"Although concurrent individual and group therapy may be necessary for some…","config":"all","split":"test","body":"{\"choices\":\"['patients beginning with a group, finding additional sources of support, tend to terminate the individual therapy prematurely', 'patients wait until they are in their individual sessions before reacting to what occurred in the group', 'screening procedures for patients may be ignored because a therapist is recommending the group experience', 'patients use their two therapists’ words to refute group members’ suggestions for behavior change']\",\"question\":\"Although concurrent individual and group therapy may be necessary for some patients, many therapists do not support it. One of the major concerns is that\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}