{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81ab8dbb-86a2-526d-8e39-3688c9d9430c","task_key":"default--test--5212","task_revision_id":"1","upstream_id":"5212","short_description":"The ecological approach to urban sociology involved studying:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the impact of city infrastructure on the local ecosystem', \\\"how men and women used the city's public spaces differently\\\", 'the way in which people organized collective protests about environmental issues', 'the rate of pollution in different parts of the city', 'how urbanization affects the local climate', 'the forms of wildlife and natural habitats that could be found on the edges of the city', 'the relationship between city size and biodiversity', 'the influence of architecture on animal populations in the city', 'how social groups colonized different areas of the city and competed for resources', 'the effects of city planning on water systems and green spaces']\",\"question\":\"The ecological approach to urban sociology involved studying:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}