{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25e4825a-13fb-529f-a553-0537b09d8f63","task_key":"default--test--5927","task_revision_id":"1","upstream_id":"5927","short_description":"The key factor Thomas Malthus failed to recognize in his population theory was","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the role of education in controlling population growth.', 'the influence of government policies on population control.', 'the influence of climate change on population sustainability.', 'the effect of migration on population distribution.', 'population is limited by their means of subsistence.', \\\"wars and famine inhibit population's reproductive capacity.\\\", 'all populations have the potential to increase more than the actual rate of increase.', \\\"technology's ability to raise the Earth's carrying capacity.\\\", 'the impact of natural disasters on population growth.', 'the impact of disease outbreaks on population size.']\",\"question\":\"The key factor Thomas Malthus failed to recognize in his population theory was\",\"src\":\"ori_mmlu-high_school_geography\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}