{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75ad484e-1a65-5dbc-8b13-72eafbc7c4a2","task_key":"all--test--75ad484e-1a65-5dbc-8b13-72eafbc7c4a2","task_revision_id":"1","upstream_id":"","short_description":"Before Justin could take an airplane flight, he needed to overcome his fear of…","config":"all","split":"test","body":"{\"choices\":\"['psychoanalysis', 'behavior therapy', 'humanistic therapy', 'existential therapy']\",\"question\":\"Before Justin could take an airplane flight, he needed to overcome his fear of flying. His therapist taught him relaxation exercises, which he practiced while first looking at pictures of airplanes, then seeing them take off at the airport, then going into an empty plane that would not take off, then finally taking a short flight. Which of the following treatments did he undergo?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}