{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01b2b998-0503-5466-9a57-47f203786a20","task_key":"default--test--5006","task_revision_id":"1","upstream_id":"5006","short_description":"The skulls of the earliest migrants to the New World do not match those of…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the early migrants were not human.', 'the early migrants did not contribute to the genetic pool of modern Native Americans.', 'the early migrants were a separate species.', 'craniometrics is an unreliable method of tracking populations.', 'skull shapes can change dramatically over generations.', 'modern Native Americans have very different DNA.', 'the findings are not significant and can be disregarded.', 'the \\\"skull\\\" haplogroup was not passed on.', 'modern Native Americans underwent a significant evolution.']\",\"question\":\"The skulls of the earliest migrants to the New World do not match those of modern Native Americans. This suggests that:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}