{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cbf08604-4b73-5985-ada7-8164f6938f7f","task_key":"all--test--cbf08604-4b73-5985-ada7-8164f6938f7f","task_revision_id":"1","upstream_id":"","short_description":"How are US 'military moms' important to a critical reading of security?","config":"all","split":"test","body":"{\"choices\":\"['Mothers are not passive objects but the specific target of military recruiters who rely on their active engagement to fuel recruitment.', \\\"Examining the role of 'military moms' reveals the extent of gendered strategizing that is required to raise and sustain a military force.\\\", 'All of these.', 'Mothers are indispensable to the exercise of military power but often overlooked in traditional security analyses.']\",\"question\":\"How are US 'military moms' important to a critical reading of security?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}