{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a3f3f3a-a7ac-542d-9c85-33c11a5de8f8","task_key":"all--test--3a3f3f3a-a7ac-542d-9c85-33c11a5de8f8","task_revision_id":"1","upstream_id":"","short_description":"During his first therapy session, a client tells Dr. Goodwill that he wants to…","config":"all","split":"test","body":"{\"choices\":\"[\\\"ethically as long as she ascertains that the client's reasons for making this request are valid.\\\", 'ethically since it is up to Dr. Goodwill to decide what to include in her therapy records.', \\\"ethically and legally as long as she includes the client's payments as income on her income tax forms.\\\", 'unethically and possibly illegally.']\",\"question\":\"During his first therapy session, a client tells Dr. Goodwill that he wants to pay for therapy in cash and doesn't want Dr. Goodwill to keep a record of the therapy sessions or his payments. If Dr. Goodwill agrees to this arrangement she has acted:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}