{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6abd491c-c00d-58e9-a0a3-f2c8bc94a8a9","task_key":"all--test--6abd491c-c00d-58e9-a0a3-f2c8bc94a8a9","task_revision_id":"1","upstream_id":"","short_description":"Which of these is an effective way of securing against cyber threats?","config":"all","split":"test","body":"{\"choices\":\"['Complete eradication of threats.', 'Ensure that the severity of cyber war as a threat is recognised, and that instances of cyber warfare are identified, publicised, and acted upon immediately.', 'All of these options.', 'Acceptance that there can be no absolute security, instead design resilient IT systems and build international cooperation and information exchange in order to reduce threat and increase resilience.']\",\"question\":\"Which of these is an effective way of securing against cyber threats?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}