{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"afc86836-5837-5891-94a9-42684ddeb6dc","task_key":"default--test--3371","task_revision_id":"1","upstream_id":"3371","short_description":"Why isn't a biome map of the earth a true representationof thevegetation…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Most of the formation boundaries of the earth are not distinct. Real vegetation types usually grade one into another.', 'Biome maps exaggerate the size of desert regions', 'Vegetation types do not merge into each other', 'Biome maps are updated too frequently to provide a stable representation', 'Biome maps fail to include aquatic vegetation formations', 'Biome maps are always accurate representations of vegetation formations', 'Biome maps represent vegetation types based on political boundaries rather than ecological characteristics', 'Biome maps only show vegetation formations in the northern hemisphere', 'The color-coding on biome maps does not correspond to actual vegetation types', 'Boundaries between formations are clearly defined']\",\"question\":\"Why isn't a biome map of the earth a true representationof thevegetation formations found on land?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}