{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"31b88710-9ed2-5bbd-8b47-80962d4ff817","task_key":"default--test--4704","task_revision_id":"1","upstream_id":"4704","short_description":"What was NOT a deciding factor in the development of Mesopotamian civilization?","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the need to form alliances with neighboring civilizations for trade and protection', 'the need to develop a calendar system for agricultural planning', 'the need to develop a complex social system that would allow the construction of canals', 'the use of irrigation to produce enough food', 'the creation of a military force for defense and conquest', 'the use of domesticated animals for labor and transportation', 'the need to construct monumental works, such as step-pyramids and temples', 'the need to concentrate population away from arable lands near rivers', 'the establishment of a centralized government for law and order', 'the need to develop a written language for record keeping and communication']\",\"question\":\"What was NOT a deciding factor in the development of Mesopotamian civilization?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}