{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f0b8d868-5a41-557e-a729-c01d623ac565","task_key":"all--test--f0b8d868-5a41-557e-a729-c01d623ac565","task_revision_id":"1","upstream_id":"","short_description":"In their study of depression, Brown & Harris (1978) found that:","config":"all","split":"test","body":"{\"choices\":\"[\\\"women's susceptibility to stress depended on their levels of social support\\\", 'self-reported depression is highest amongst men and the relatively affluent', 'men and women have fairly similar levels of diagnosed depression', 'men are more vulnerable to depression because of conventional gender roles']\",\"question\":\"In their study of depression, Brown & Harris (1978) found that:\",\"subject\":\"sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}