{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce84da70-efdd-5bcf-a189-7206eed7928a","task_key":"all--test--ce84da70-efdd-5bcf-a189-7206eed7928a","task_revision_id":"1","upstream_id":"","short_description":"Which of these statements characterizes the traditional concept of national…","config":"all","split":"test","body":"{\"choices\":\"['All of these options.', 'In the national security approach, real threats are only those posed to human life, human rights, or human dignity.', 'The traditional interpretations of security and threats are important boundaries that define human security.', 'The concept of national security is too narrow to be useful when analysing the broad threats that currently exist.']\",\"question\":\"Which of these statements characterizes the traditional concept of national security, as opposed to human security?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}