{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d69cd412-ee4f-55a9-a258-9a3a77ce9734","task_key":"all--test--d69cd412-ee4f-55a9-a258-9a3a77ce9734","task_revision_id":"1","upstream_id":"","short_description":"According to West, which of the following reasons for a women to engage in sex…","config":"all","split":"test","body":"{\"choices\":\"['She or her children are economically dependent on her male partner, and so she reasons that she better have sex with him and keep him happy.', \\\"Her refusal will result in her partner's being in a foul mood which is more burdensome than having undesired sex.\\\", 'Her refusal might result in a violent outburst.', 'All of the above.']\",\"question\":\" According to West, which of the following reasons for a women to engage in sex with her male partner may cause her harm?\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}