{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1a59981-1500-519c-982a-ad96cd9793b8","task_key":"default--test--4971","task_revision_id":"1","upstream_id":"4971","short_description":"The Aztec Empire was based on the systematic expansion of:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['a vast network of roads and trade routes.', 'the expansion and control of waterways for transportation and irrigation.', 'the cultivation and trade of cacao and maize.', 'religious beliefs that required extensive and escalating human sacrifice.', 'tribute in the form of gold, jade, feathers, cloth, and jewels.', 'the establishment of colonies in unexplored territories.', 'ceramic production, a distinctive architectural style, and artistic expression.', 'farming techniques such as terracing and irrigation.', 'a strict caste system that promoted social stratification.', 'oppressive social and religious control based on military conquest.']\",\"question\":\"The Aztec Empire was based on the systematic expansion of:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}