{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa41e806-6c95-597e-b019-654fe562d370","task_key":"all--test--fa41e806-6c95-597e-b019-654fe562d370","task_revision_id":"1","upstream_id":"","short_description":"Why do so many feminist scholars regard the exclusion of women from the military…","config":"all","split":"test","body":"{\"choices\":\"['Veterans of armed conflict may be rewarded by gaining powerful offices, often in political positions and in state institutions.', 'Participation in military activity is often regarded as a sign of strength, patriotism and courage.', \\\"The foundations for decisions not to allow female participation are commonly based on arbitrary and discriminatory rationale that can limit women's participation in civil society in other ways.\\\", 'All of these options.']\",\"question\":\"Why do so many feminist scholars regard the exclusion of women from the military sphere or from armed combat as important?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}