{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0574cbca-c5da-5192-b0d0-474a4a2c17bb","task_key":"all--test--0574cbca-c5da-5192-b0d0-474a4a2c17bb","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Mali was able to forge advanced bronze tools from its mineral resources.', 'International trade with Sub-Saharan Africa involved the exchange of mineral resources.', 'The jizya could only be collected on unbelievers who had minerals to exploit.', \\\"Mali's relatively advanced mining technologies accounted for its mineral wealth.\\\"]\",\"question\":\"This question refers to the following information.\\n\\\"Al-Zawawi also said 'This sultan Musa told me that at a town called ZKRY he has a copper mine from which ingots are brought to BYTY. \\\"There is nothing in my kingdom, Musa said, on which a duty is levied except this crude copper which is brought in. Duty is collected on this and on nothing else. We send it to the land of the pagan Sudan and sell it for two-thirds of its weight in gold, so that we sell 100 measures of this copper for 66 2/3 measures of gold.' He also stated that there are pagan nations in his kingdom from whom he does not collect the tribute (jizya) but whom he simply employs in extracting the gold from its deposits. The gold is extracted by digging pits about a man's height in depth and the gold is found embedded in the sides of the pits or sometimes collected at the bottom of them.\\\"\\nFrom N. Levtzion & J.F.P. Hopkins, eds. Corpus of Early\\nArabic Sources for West African History. Cambridge\\nUniversity Press, 1981. Reprinted with permission of\\nCambridge University Press.\\nAl' Umari, ca. 1300–1384\\nWhich of the following inferences is most supported by the discussion of copper and gold in the passage?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}