{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e6b6f0b-6537-5d97-89f7-d4271e047469","task_key":"default--test--3369","task_revision_id":"1","upstream_id":"3369","short_description":"Describe the various land biomes that are usually encounteredby a traveler going…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Tropical rain forests, savannas, and tundra', 'Tropical rain forests, deserts, and polar ice caps only', 'Tundra, desert regions, and tropical rain forests', 'Taiga biome, grasslands, and deserts', 'Deciduous forests, tundra, and savannas', 'Deciduous forests, grasslands, and tropical savannas', 'Savannas, taiga biome, and polar ice caps', 'Chaparral, wetlands, and taiga biome', 'Tropical rain forests, montane forests, and arctic tundra', 'The various land biomes encountered while traveling from the equator to the arctic polar ice cap are tropical rain forests, grassland biomes, deciduous forests, taiga biome, tundra, and desert regions.']\",\"question\":\"Describe the various land biomes that are usually encounteredby a traveler going from the equator to the arcticpolar ice cap.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}