{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d1ca0c24-fd14-55c4-bf95-4b702207b57b","task_key":"all--test--d1ca0c24-fd14-55c4-bf95-4b702207b57b","task_revision_id":"1","upstream_id":"","short_description":"A psychologist is the supervisor of several interns at a community hospital in a…","config":"all","split":"test","body":"{\"choices\":\"['unethically by agreeing to get involved in a dual relationship.', \\\"unethically by violating the intern''s right to privacy.\\\", 'ethically because they are working in a community hospital in a small town.', 'ethically because the intern is in desperate need of help.']\",\"question\":\"A psychologist is the supervisor of several interns at a community hospital in a small town and notices that one of the interns seems unenthusiastic and depressed. When he talks to her, she tells him that she’s been very unhappy lately and feels she has no one to turn to. The intern asks the psychologist to see her in therapy, and he agrees to do so. The psychologist has acted:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}