{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a3a1674d-db00-522a-b1f7-df3385e6b9f3","task_key":"all--test--a3a1674d-db00-522a-b1f7-df3385e6b9f3","task_revision_id":"1","upstream_id":"","short_description":"Considering 'offensive' and 'defensive realism', which of these statements is…","config":"all","split":"test","body":"{\"choices\":\"['Offensive realism defines the international system as an environment in which states seek to achieve security through the acquisition of power.', 'Defensive realism regards cooperation as the way in which states can achieve security.', 'Both defensive realism and offensive realism regard power and competition as a means by which security can be achieved for the state.', 'Offensive realism theorizes that states will pursue hegemony where possible.']\",\"question\":\"Considering 'offensive' and 'defensive realism', which of these statements is not correct?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}