{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0cfdb2a2-ce63-56e8-b407-6dac34560d60","task_key":"default--test--6054","task_revision_id":"1","upstream_id":"6054","short_description":"Which of the following is not a reason for increased risk of vitamin or mineral…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Regular physical activity and outdoor exposure', 'High nutrient requirements for tissue turnover', 'Increased intrinsic factor in the stomach', 'Low income so decreased ability to purchase nutrient-rich foods', 'Decreased intrinsic factor in the stomach', 'Decreased nutrient requirements due to aging', 'Decreased mobility and little time spend outdoors in the sunshine', 'Increased absorption of vitamins and minerals due to medication']\",\"question\":\"Which of the following is not a reason for increased risk of vitamin or mineral deficiencies among older people in high income countries?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}