{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce4051d3-eba4-5277-a4d1-833509333b33","task_key":"default--test--5013","task_revision_id":"1","upstream_id":"5013","short_description":"Among the Inca, human sacrifice was preceded by:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['elite-sponsored feasts in which captives of war were beheaded.', 'public humiliation, forced labor, and consumption of hallucinogenic plants.', 'ritualistic battles, consumption of coca leaves, and bloodletting.', 'communal feasts and the torture of captives to be sacrificed.', 'improved diet, forced labor, and the use of coca leaves.', 'ritual fasting, solitary confinement, and the use of psychedelic substances.', 'physical punishment, fasting, and ritualistic dances.', 'elite-sponsored feasts where captives of war were forced to fight to death.', 'ritual starvation, torture, and blood sacrifice.', 'improved diet, the use of coca leaves, and consumption of maize beer.']\",\"question\":\"Among the Inca, human sacrifice was preceded by:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}