{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad9c9f8c-b70c-5eff-be68-c70b25877d44","task_key":"all--test--ad9c9f8c-b70c-5eff-be68-c70b25877d44","task_revision_id":"1","upstream_id":"","short_description":"Wayne W. tells Dr. Pete Pannick that his partner is threatening to harm Dr.…","config":"all","split":"test","body":"{\"choices\":\"['should continue to see Wayne in therapy and ask Wayne to bring his partner to the next session to discuss the situation.', 'should continue seeing Wayne in therapy but, with his permission, contact the police to discuss the threat.', 'may terminate therapy with Wayne but only after providing him with pre-termination counseling and appropriate referrals.', 'may terminate therapy with Raymond.']\",\"question\":\"Wayne W. tells Dr. Pete Pannick that his partner is threatening to harm Dr. Panniclc because he's not happy with the advice that Wayne is getting in therapy. Dr. Panniclc knows that Wayne's partner has engaged in violent behavior in the past and believes the threat is serious. Dr. Panniclc:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}