{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"09beb7d1-64c9-5ae4-ba6f-674be0221a2e","task_key":"all--test--09beb7d1-64c9-5ae4-ba6f-674be0221a2e","task_revision_id":"1","upstream_id":"","short_description":"Which of these statements cannot be applied to the initial development of peace…","config":"all","split":"test","body":"{\"choices\":\"['Mathematicians and natural scientists entered the field, which was otherwise a social-sciences area of study, adding to the interdisciplinary nature of the field.', 'Now of fundamental importance to peace studies as a field of research, the creation of journals was not a prominent feature of the peace studies until late in the twentieth century.', 'Most early work was carried out in North America and Scandinavia.', 'The establishment of research institutes across the world has been one of the lasting successes of peace studies.']\",\"question\":\"Which of these statements cannot be applied to the initial development of peace studies?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}