{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97a22b44-f842-5315-a9b9-63720524b8ac","task_key":"all--test--97a22b44-f842-5315-a9b9-63720524b8ac","task_revision_id":"1","upstream_id":"","short_description":"In terms of cyber-security, what is meant by 'resilience' and why is it…","config":"all","split":"test","body":"{\"choices\":\"['Resilience refers to the ability of an IT system to identify and defend against a cyber-security threat.', 'The only way to build a truly resilient IT system is to ensure that it can repel all types of threat and remain and avoid attack.', 'Resilience in an IT system refers to its ability to function when there are no threats to its security.', 'Resilience refers to the ability of IT systems to recover from attack.']\",\"question\":\"In terms of cyber-security, what is meant by 'resilience' and why is it important?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}