{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89c8f373-0396-5561-8933-2e92b6a4cdb8","task_key":"default--test--4926","task_revision_id":"1","upstream_id":"4926","short_description":"According to one theory, the power of Ancestral Pueblo elites emerged when…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['created intricate pottery and jewelry.', 'discovered the use of fire.', 'invented the wheel.', 'stored food surpluses in good years for distribution in bad years.', 'established trade routes with distant cultures.', 'constructed monuments and elaborate multi-storied cliff dwellings.', 'introduced advanced farming techniques.', 'predicted astronomical events such as eclipses and meteor showers.', 'incorporated other societies through conquest warfare.', 'developed a written language.']\",\"question\":\"According to one theory, the power of Ancestral Pueblo elites emerged when certain groups:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}