{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d1639c5c-7144-551e-913d-ef39adf494e8","task_key":"all--test--d1639c5c-7144-551e-913d-ef39adf494e8","task_revision_id":"1","upstream_id":"","short_description":"You regularly waive the co-payment for your low-income clients who are covered…","config":"all","split":"test","body":"{\"choices\":\"['a common and ethical practice.', \\\"ethical as long as you don't bill the insurance company for an increased hourly fee in order to collect your full fee.\\\", 'ethical as long as the insurance company does not explicitly prohibit such arrangements.', 'ethical only if the insurance company is aware of the arrangement.']\",\"question\":\"You regularly waive the co-payment for your low-income clients who are covered by insurance and believe this is acceptable since you normally bill your clients on a sliding fee scale. This is:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}