{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1f160e35-c266-598a-9af7-c7b147fd83cf","task_key":"all--test--1f160e35-c266-598a-9af7-c7b147fd83cf","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['European merchants were confined to a few cities designated for foreign trade.', 'Only the Dutch traded with China; other Europeans had to use them as intermediaries.', 'The Chinese imposed extraordinary tariffs on all foreign products.', 'They passed laws intended to persecute and harass foreign residents.']\",\"question\":\"This question refers to the following information.\\nThe passage below is the Chinese emperor's response to English King George III's diplomatic envoys, who were seeking expanded trading privileges (1793).\\nStrange and costly objects do not interest me. If I have commanded that the tribute offerings sent by you, O King, are to be accepted, this was solely in consideration for the spirit which prompted you to dispatch them from afar. . . . As your Ambassador can see for himself, we possess all things. I set no value on objects strange or ingenious, and have no use for your country's manufactures. It behooves you, O King, to display even greater devotion and loyalty in future, so that, by perpetual submission to our Throne, you may secure peace and prosperity.\\nHow did the Chinese restrict foreign trade during the era 1750–1900?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}