{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d729cda5-8504-5aa0-9423-b4e68d32789e","task_key":"all--test--d729cda5-8504-5aa0-9423-b4e68d32789e","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Adroit diplomacy and establishment of client relationships with bordering nomads', 'The ability of the Tang emperor to project military power on the frontier in order to impose his will', 'The permanent settlement of Han Chinese in frontier zones', 'The payment of tribute to nomadic rulers']\",\"question\":\"This question refers to the following information.\\nI walk alongside the column, ask what's going on.\\nA soldier says simply: \\\"They call up more every day.\\n\\\"Some of us were sent north to the Yellow River at age fifteen,\\nAnd now at forty we're heading off to the garrisons in the west.\\nOn our first tour, the village headman had to tie our bandannas for us.\\nWhen we came back, our hair was white, but still there's more unrest.\\nThe frontier garrisons run with blood, enough to fill an ocean,\\nBut the Martial Emperor's territorial ambitions have yet to crest.\\nIn the hundred districts east of the mountains, throughout the land of Han,\\nThere must be ten thousand villages that brambles now infest.\\nEven if wives are strong enough to handle a hoe and plow,\\nThe crops grow every which way, the fields are all a mess.\\nIt's hardest for the Shanxi men, with their reputations as fighters:\\nThey're rounded up like dogs or chickens, every male impressed.\\n\\\"But sir, though it's good of you to ask,\\nComplaining isn't part of the soldier's task.\\nWe can only shake our heads. Take this winter:\\nThe Shanxi troops were never sent home.\\nThe District Officers are demanding the land tax,\\nBut where will it come from? You can't get blood from a stone!\\nI honestly think it's bad luck to bear a son now,\\nIt's better to have a daughter: at least she can marry\\nAnd live with the neighbors next door.\\nBut a son will end up lying on some distant prairie.\\\"\\nDu Fu, \\\"Ballad of the Army Carts,\\\" ca. 750 C.E.\\nBased on the poem and your knowledge of history, Tang relations with western nomadic peoples and frontier peoples relied on which of the following?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}