{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"104dc06e-822d-5b77-a5a7-8f8c02e7c00f","task_key":"default--test--5660","task_revision_id":"1","upstream_id":"5660","short_description":"Prior to the late 20th century the role of women in war had been neglected as an…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Women did not actively engage in warfare as combatants and so questions of gender or the role of women were not important in considerations of security.', 'The roles that women played in violence, be it as victims, perpetrators or otherwise, was framed in terms of what effect their roles had on the state, rather than as individuals or as women.', 'The role of women was given appropriate consideration; women have fulfilled many vital roles in the support of the dominant security system.', 'Women are generally unsupportive of warfare and prefer to engage in anti-war or opposition movements, rendering their contribution to the study of warfare as marginal.']\",\"question\":\"Prior to the late 20th century the role of women in war had been neglected as an area of concern in security studies, for what reason?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}