{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f2f38f1-bcbd-558a-807f-2d090b34ae8f","task_key":"default--test--6384","task_revision_id":"1","upstream_id":"6384","short_description":"Since the 1996 World Food Summit, how has the number of food insecure people in…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The number doubled due to increases in global population.', \\\"The number remained the same despite the Summit's efforts.\\\", \\\"The number decreased, but not by nearly enough to meet the Summit's goal.\\\", 'The number fluctuated wildly due to unpredictable factors like climate change and political unrest.', \\\"The Summit's goal of cutting the number of hungry people in half by 2015 was achieved.\\\", 'The number increased moderately due to economic instability in certain regions.', \\\"The number decreased dramatically due to successful implementation of the Summit's strategies.\\\", 'The number reduced to zero due to global cooperation in food production and distribution.', 'Because of rising food prices, the number increased dramatically.', 'The number increased slightly.']\",\"question\":\"Since the 1996 World Food Summit, how has the number of food insecure people in the world changed?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}