{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"747d6ac4-5506-58fa-a393-6798897a9ead","task_key":"default--test--5870","task_revision_id":"1","upstream_id":"5870","short_description":"Margaret Mead suggested that:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['cultural norms have no influence on adolescence and gender roles', 'gender roles in three New Guinea societies were identical and so must be biologically determined', 'anthropological fieldwork is completely objective and unbiased', \\\"anthropological fieldwork can be problematic because the researcher's values affect the way they interpret their observations\\\", 'gender roles are universally identical and biologically determined', 'every society has the same adolescence experience', 'adolescence and gender roles varied between societies and so were culturally determined', 'there are no variations in gender roles and adolescence across different cultures', 'adolescence in the USA is a time of relative calm compared to the experience in Japan and Europe', 'adolescence is universally a time of turmoil and confusion']\",\"question\":\"Margaret Mead suggested that:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}