{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6881e810-074c-5434-bc8c-5e2df6295f45","task_key":"default--test--4911","task_revision_id":"1","upstream_id":"4911","short_description":"Pizarro was able to subdue the entire Inca empire largely because:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the Incas were suffering from a plague when Pizarro arrived.', 'the empire had never won the allegiance of the peoples it had conquered.', 'the Incas were already weakened by a civil war.', 'the Incas were expecting Pizarro and willingly surrendered to him.', 'he arrived with a superior force of more than 80,000 soldiers.', 'Pizarro used advanced technology that the Incas could not comprehend.', 'his horses and guns frightened the Incas into submission.', 'Pizarro had the support of other native tribes who were enemies of the Incas.', 'he was such a brilliant military strategist.', 'Pizarro manipulated the Incas into believing he was a god.']\",\"question\":\"Pizarro was able to subdue the entire Inca empire largely because:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}