{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e82f8f3e-0f94-512b-9e7e-6b0f52bb81dd","task_key":"all--test--e82f8f3e-0f94-512b-9e7e-6b0f52bb81dd","task_revision_id":"1","upstream_id":"","short_description":"Prior to the late 20th century the role of women in war had been neglected as an…","config":"all","split":"test","body":"{\"choices\":\"['Women did not actively engage in warfare as combatants and so questions of gender or the role of women were not important in considerations of security.', 'The roles that women played in violence, be it as victims, perpetrators or otherwise, was framed in terms of what effect their roles had on the state, rather than as individuals or as women.', 'The role of women was given appropriate consideration; women have fulfilled many vital roles in the support of the dominant security system.', 'Women are generally unsupportive of warfare and prefer to engage in anti-war or opposition movements, rendering their contribution to the study of warfare as marginal.']\",\"question\":\"Prior to the late 20th century the role of women in war had been neglected as an area of concern in security studies, for what reason?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}