{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bda02952-937c-5b2b-a541-2085123a0d47","task_key":"all--test--bda02952-937c-5b2b-a541-2085123a0d47","task_revision_id":"1","upstream_id":"","short_description":"You have been seeing Leticia Lopez in therapy for several months. Leticia is 24…","config":"all","split":"test","body":"{\"choices\":\"['Give Mrs. Lopez the specific advice she has requested.', 'Tell Mrs. Lopez to ask Leticia what she (Mrs. Lopez) can do to help her.', 'Tell Mrs. Lopez that it would be best if you discussed this matter with Leticia.', 'suggest that Mrs. Lopez accompany Leticia to her next therapy session.']\",\"question\":\"You have been seeing Leticia Lopez in therapy for several months. Leticia is 24 years old and lives with her widowed mother, who is paying for Leticia’s therapy. One day, Leticia's mother calls and says she is very concerned about Leticia and wants to know what she can do to help Leticia feel better about herself. Mrs. Lopez asks that you not tell Leticia that she has called. Your best course of action in this situation would be to:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}