{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"08c520e1-187e-505a-a577-391a83f23b4d","task_key":"default--test--4786","task_revision_id":"1","upstream_id":"4786","short_description":"Which is true of the origins of Minoan civilization?","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Minoan civilization was the result of a merger between ancient Greek and Persian cultures.', 'Crete was colonized by the Egyptians, who called the island \\\"Minoa.\\\"', 'Minoan culture was a direct offshoot of the ancient Sumerian civilization.', 'The Minoans were a splinter group of the ancient Indus Valley Civilization.', 'Early humans crossed a land bridge to Crete around 150,000 B.P. and gradually became Minoans.', 'The Minoans were descendants of the ancient Chinese who migrated to Crete.', 'Crete was originally populated by the Romans, who then developed into the Minoans.', 'Cretan culture was a mix of Egyptian and Mesopotamian elements.', 'Crete was populated by Greeks and others, but Minoan culture was a local development.', 'Minoan civilization was brought to Crete by the Phoenicians.']\",\"question\":\"Which is true of the origins of Minoan civilization?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}