{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5abcd06a-5c33-5ba2-82e4-cadeb054c27b","task_key":"all--test--5abcd06a-5c33-5ba2-82e4-cadeb054c27b","task_revision_id":"1","upstream_id":"","short_description":"Identify the conclusion of the following argument. It is hard not to verify in…","config":"all","split":"test","body":"{\"choices\":\"['It is hard not to verify in our peers the same weakened intelligence due to emotions that we observe in our everyday patients.', 'The arrogance of our consciousness, which in general, belongs to the strongest defense mechanisms, blocks the unconscious complexes.', 'Because of this, it is difficult to convince people of the unconscious, and in turn to teach them what their conscious knowledge contradicts.', 'It is difficult to convince people of the unconscious, and in turn to teach them what their conscious knowledge contradicts.']\",\"question\":\"Identify the conclusion of the following argument. It is hard not to verify in our peers the same weakened intelligence due to emotions that we observe in our everyday patients. The arrogance of our consciousness, which in general, belongs to the strongest defense mechanisms, blocks the unconscious complexes. Because of this, it is difficult to convince people of the unconscious, and in turn to teach them what their conscious knowledge contradicts. (Sigmund Freud, The Origin and Development of Psychoanalysis)\",\"subject\":\"formal_logic\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}