{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d97318e4-312e-5305-af2b-0dd018eba800","task_key":"default--test--4867","task_revision_id":"1","upstream_id":"4867","short_description":"Unlike Mesopotamian writings, the records left by the Maya are principally…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['music and arts.', 'geography and climate.', 'architecture and construction methods.', 'legal and judicial systems.', 'political and military history.', 'philosophy and religion.', 'trade and economics.', 'social and cultural practices.', 'agriculture and farming techniques.']\",\"question\":\"Unlike Mesopotamian writings, the records left by the Maya are principally concerned with:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}