{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a273fc66-6442-5944-8048-7d91bf8f07fc","task_key":"default--test--5195","task_revision_id":"1","upstream_id":"5195","short_description":"Which one of the following is NOT an advantage of urban agriculture?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Beautification of a dingy urban area', 'Improved air quality', 'Increase in property values', 'Helping to solve the problem of solid waste disposal', 'Increased urban biodiversity', 'Reduction in carbon footprint', 'Renewed or purified water supplies', 'Fresh produce for sale to others', 'Reduced dependence on fossil fuels']\",\"question\":\"Which one of the following is NOT an advantage of urban agriculture?\",\"src\":\"ori_mmlu-high_school_geography\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}