{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c83dd740-6a37-5f24-ac5b-f981caa2f428","task_key":"all--test--c83dd740-6a37-5f24-ac5b-f981caa2f428","task_revision_id":"1","upstream_id":"","short_description":"A former client owes you over $600.00 in therapy fees and you are considering…","config":"all","split":"test","body":"{\"choices\":\"['decide not to do so because using a collection agency is prohibited by the ethical guidelines.', 'do so only if you had informed the client at the beginning of therapy that you would do so if she did not pay her fees in a timely manner.', 'contact the client first to inform her of your intent to use a collection agency if she does not pay her outstanding fees by a specific date.', 'use a collection agency only as a “last resort.”']\",\"question\":\"A former client owes you over $600.00 in therapy fees and you are considering using a collection agency to collect the money she owes you. As an ethical psychologist, you should:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}