{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"507f11f1-8913-5c8c-95c6-ecb24caa66fb","task_key":"all--test--507f11f1-8913-5c8c-95c6-ecb24caa66fb","task_revision_id":"1","upstream_id":"","short_description":"You receive a phone call from Hermann H., age 28, who says he is “totally…","config":"all","split":"test","body":"{\"choices\":\"['consult with another psychologist during the course of treatment to make sure your feelings don’t interfere with your objectivity.', 'discuss the difference in political views with Hermann only if they become relevant to the psychotherapy process.', 'see Hermann in therapy until his current crisis is over and then make a referral if necessary.', 'provide Hermann with appropriate referrals.']\",\"question\":\"You receive a phone call from Hermann H., age 28, who says he is “totally miserable” because of the recent breakup with his girlfriend and that he would like to begin therapy with you. During the first session with Hermann, you find out that his political views are completely repugnant to you, and you feel that you would not enjoy working with him. As an ethical psychologist, you should:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}