{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e440ac5e-99ad-5e73-bfa9-4d9adbea4267","task_key":"default--test--5516","task_revision_id":"1","upstream_id":"5516","short_description":"Marxist feminists explain patriarchy in terms of:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the capitalist system being inherently biased against women', \\\"the exploitation of women's emotional labour by the capitalist system\\\", 'societal norms and traditions that favour men', 'the inherent biological differences between men and women', 'a lack of equal rights and opportunities for men and women', \\\"women's domestic labour being exploited by the capitalist economy\\\"]\",\"question\":\"Marxist feminists explain patriarchy in terms of:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}