{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdface98-a41e-5ebd-9555-423f26870bbb","task_key":"default--test--5027","task_revision_id":"1","upstream_id":"5027","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The Zulu lands offered fertile soil for agricultural activities.', 'A gold rush ensued following the discovery of gold on Zulu lands.', 'The British government offered tax incentives to citizens moving to Zulu lands.', 'The Zulu created attractive trade ports along the Indian Ocean.', 'A diamond rush ensued following the discovery of diamonds on Zulu lands.', 'The British military offered land grants to soldiers who would settle in the region.', 'Europeans were attracted to the rich cultural heritage of the Zulu lands.', 'The British crown offered incentives to those who would establish homesteads in the unsettled lands of South Africa.', 'The British government offered free passage to citizens willing to migrate to Zulu lands.', 'The Berlin Conference clarified boundaries, thus making it logistically sound for Europeans to move to the African colonies.']\",\"question\":\"This question refers to the following information.\\n\\\"My little homestead in the city, which I recently insured for ￡2,000 would no doubt have shared the common fate, as the insurance companies will not make good that which is destroyed by the Queen's enemies. And although I have a farm of 50 acres close to the town, no doubt the crops and premises would have been destroyed. In fact, this has already partly been the case, and I am now suing the Government for damages done by a contingent of 1,500 natives that have recently encamped not many hundred yards from the place, who have done much damage all around.\\\"\\nLetter from a British citizen to his sister during the Anglo-Zulu War, South Africa, 1879\\nWhich of the following reasons explains why European citizens moved in large numbers to Zulu lands in 1867?\",\"src\":\"ori_mmlu-high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}