{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f05c71b9-d919-5fa6-98a5-a3e868e69dc7","task_key":"all--test--f05c71b9-d919-5fa6-98a5-a3e868e69dc7","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The strain placed on merchant families by long periods of separation', \\\"Religious disagreements arising from the merchant's exposure to Hinduism\\\", 'A decision on the part of the merchant to remain permanently in India', 'Bankruptcy stranding the merchant far from home']\",\"question\":\"This question refers to the following information.\\nNow if divorce is your wish, I cannot blame you. For the waiting has been long. And I do not know whether the Creator will grant relief immediately so that I can come home, or whether matters will take time, for I cannot come home with nothing. Now the matter is in your hand. If you wish to end our marriage, accept this bill of repudiation, and you are free. May God inspire you with the right decision.\\nLetter from a Jewish trader in India to his wife in Cairo, ca. 1200\\nWhich of the following is the most probable cause of the marital strain evident in the passage above?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}