{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2927a44-7a9a-5dc6-8362-8243e6badefe","task_key":"all--test--b2927a44-7a9a-5dc6-8362-8243e6badefe","task_revision_id":"1","upstream_id":"","short_description":"In a cartoon, an angel is perched on one shoulder and a devil is perched on the…","config":"all","split":"test","body":"{\"choices\":\"['libido', 'id', 'ego', 'superego']\",\"question\":\"In a cartoon, an angel is perched on one shoulder and a devil is perched on the other shoulder of a character who needs to decide whether to give money to a homeless man or not. The devil says, \\\"Don't give him anything, you'll make better use of the money than he will.\\\" The angel says, \\\"Give him the money because he needs it more than you do.\\\" The character gives the homeless man half of his money. In a Freudian interpretation, the angel represents the character's\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}