{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6b5487f-5d88-5eb7-bac5-5da17ba45bd1","task_key":"default--test--6130","task_revision_id":"1","upstream_id":"6130","short_description":"Which of the following could lead to iodine deficiency?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Living at high altitudes', 'Excessive consumption of fast food', 'Intake only of locally grown food in central Africa', 'Low intake of dairy products', 'Drinking too much water', 'Lack of physical activity', 'Low intake of fruits and vegetables', 'Inadequate sunlight exposure', 'Insufficient caloric intake', 'Overconsumption of caffeinated beverages']\",\"question\":\"Which of the following could lead to iodine deficiency?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}