{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b718f40e-7ead-5f05-9fab-49f8bb619bfd","task_key":"default--test--5005","task_revision_id":"1","upstream_id":"5005","short_description":"Which of the following is archaeological evidence of social stratification and…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Burials with exotic grave goods also contain bones indicating that they engaged in more strenuous labor.', 'Burials all contain equal amounts of exotic grave goods, but some are more impressive than others.', 'Burials without exotic grave goods also contain bones indicating that they engaged in strenuous labor.', 'Burials all contain bones indicating that they engaged in same level of strenuous labor.', 'Burials contain bones indicating that people were segregated based on gender, stature, and personal capabilities, including artistic abilities.', 'All burials contain equal amounts of non-exotic grave goods.', 'Burials with exotic grave goods also contain bones indicating that they engaged in less strenuous labor.', 'Burials contain bones indicating that people were segregated based on their age and health.', 'Burials without exotic grave goods also contain bones indicating that they engaged in less strenuous labor.', \\\"Burials contain only exotic grave goods with no indication of the deceased's labor.\\\"]\",\"question\":\"Which of the following is archaeological evidence of social stratification and inequality?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}