{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6539f295-75af-5061-8051-3cc9ceb1b0d0","task_key":"default--test--4706","task_revision_id":"1","upstream_id":"4706","short_description":"What do the amounts of rainfall in the Maya homeland, as reflected in variations…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['severe drought and a surge in warfare', 'increased rainfall and population migration to the Maya ceremonial centers', 'severe drought and increased storage of surplus food from agriculture', 'severe drought and a decline in construction at Maya ceremonial centers', 'increased warfare and the abandonment of the entire region', 'increased rainfall and increased storage of surplus food from agriculture', 'increased rainfall and severe flooding that led to the collapse of Maya ceremonial centers', 'increased warfare and increased construction at Maya ceremonial centers', 'decreased rainfall and a rise in the construction of Maya ceremonial centers', 'decreased rainfall and increased storage of surplus food from agriculture']\",\"question\":\"What do the amounts of rainfall in the Maya homeland, as reflected in variations in oxygen isotopes in a stalagmite in Yok Balum Cave in Belize, correspond to?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}