{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e677bb8e-829f-5ba6-afab-4a1cbe04112a","task_key":"all--test--e677bb8e-829f-5ba6-afab-4a1cbe04112a","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The formation of far-flung diaspora communities as a result of interregional commerce', 'The continued use of organized religion to justify patriarchy', 'The permanent division of families due to armed conflict along established trade routes', 'The evolution of family structure among polytheistic populations']\",\"question\":\"This question refers to the following information.\\nNow if divorce is your wish, I cannot blame you. For the waiting has been long. And I do not know whether the Creator will grant relief immediately so that I can come home, or whether matters will take time, for I cannot come home with nothing. Now the matter is in your hand. If you wish to end our marriage, accept this bill of repudiation, and you are free. May God inspire you with the right decision.\\nLetter from a Jewish trader in India to his wife in Cairo, ca. 1200\\nThe passage above is best seen in light of which of the following historical developments?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}