{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c95447d3-c161-5da3-9806-a41761e40bb7","task_key":"default--test--4672","task_revision_id":"1","upstream_id":"4672","short_description":"What did the Moche build at the heart of their urban center?","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the Temple of the Feathered Serpent', 'an enormous walled-in precinct of elite residences', 'the Palace of the Painted Walls', 'a complex irrigation system', 'the Pyramid of the Sun', 'the Tower of Moche', 'an extensive library of their written records', 'a central marketplace for trade', 'a vast complex of temples and artisan workshops', 'an amphitheater for gladiatorial combats']\",\"question\":\"What did the Moche build at the heart of their urban center?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}