{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af12529a-df1f-55f4-bf5f-69620ada8c00","task_key":"default--test--4707","task_revision_id":"1","upstream_id":"4707","short_description":"What do we know about the people who built Stonehenge based on the analysis of…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['They had no domesticated animals but hunted wild animals for meat.', 'They had domesticated animals, but they did not consume their meat.', 'They bred and raised a specific species of deer for food.', 'They were vegetarian and did not consume meat.', 'They brought animals to the site from outside of the region.', 'They raised livestock at the site, including cattle, sheep, and goats.', 'They hunted antelope, gazelle, and bear.', 'They imported exotic animals from distant lands.', 'They primarily raised and hunted aquatic animals.', 'They hunted only large game animals, such as mammoths and bison.']\",\"question\":\"What do we know about the people who built Stonehenge based on the analysis of animal teeth from a nearby village?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}