{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9544ad2d-d657-5f21-b12d-688e034b0ece","task_key":"all--test--9544ad2d-d657-5f21-b12d-688e034b0ece","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The British crown offered incentives to those who would establish homesteads in the unsettled lands of South Africa.', 'The Zulu created attractive trade ports along the Indian Ocean.', 'The Berlin Conference clarified boundaries, thus making it logistically sound for Europeans to move to the African colonies.', 'A diamond rush ensued following the discovery of diamonds on Zulu lands.']\",\"question\":\"This question refers to the following information.\\n\\\"My little homestead in the city, which I recently insured for ￡2,000 would no doubt have shared the common fate, as the insurance companies will not make good that which is destroyed by the Queen's enemies. And although I have a farm of 50 acres close to the town, no doubt the crops and premises would have been destroyed. In fact, this has already partly been the case, and I am now suing the Government for damages done by a contingent of 1,500 natives that have recently encamped not many hundred yards from the place, who have done much damage all around.\\\"\\nLetter from a British citizen to his sister during the Anglo-Zulu War, South Africa, 1879\\nWhich of the following reasons explains why European citizens moved in large numbers to Zulu lands in 1867?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}