{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d0fcc58-b747-58d2-a68b-2897c6f6eef4","task_key":"all--test--3d0fcc58-b747-58d2-a68b-2897c6f6eef4","task_revision_id":"1","upstream_id":"","short_description":"Emic approaches to multicultural counseling training are often advocated on the…","config":"all","split":"test","body":"{\"choices\":\"['difficulty in gaining knowledge regarding a large number of cultural groups', \\\"the presence of many members of a particular cultural group in the training program's catchment area\\\", 'the need to focus on personal and cultural meaning for each individual client', 'the need to view culturally different groups in similar ways']\",\"question\":\"Emic approaches to multicultural counseling training are often advocated on the basis of\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}