{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a41faaaa-7a50-5e6e-be07-3e9df15a9883","task_key":"all--test--a41faaaa-7a50-5e6e-be07-3e9df15a9883","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['wandering Berber mystics.', 'conquering Mughal armies.', 'contact with Muslim trade caravans.', 'pilgrims to Islamic shrines in Ethiopia.']\",\"question\":\"This question refers to the following information.\\nAs for their men there is no sexual jealousy in them. And none of them derives his genealogy from his father but, on the contrary, from his maternal uncle. A man does not pass on inheritance except to the sons of his sister to the exclusion of his own sons. . . . They are Muslims keeping to the prayers, studying fiqh (Islamic jurisprudence) and learning the Qur'an by heart. With regard to their women, they are not modest in the presence of men; they do not veil themselves in spite of their perseverance in the prayers. He who wishes to marry among them can marry, but the women do not travel with the husband, and if one of them wanted to do that, she would be prevented by her family. The women there have friends and companions amongst men outside the prohibited degrees of marriage [i.e., other than brothers, fathers, etc.]. Likewise for the men, there are companions from amongst women outside the prohibited degrees. One of them would enter his house to find his wife with her companion and would not disapprove of that conduct.\\n—Adapted from an account by Ibn Battuta of his travels in Mali during the 1300s\\nThe most likely source of Islam in Mali came from\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}