{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d45bd19-8352-5e91-9619-c73a6df5b8c5","task_key":"all--test--1d45bd19-8352-5e91-9619-c73a6df5b8c5","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The movement of many agricultural workers to the new industrial cities increased urbanization.', 'Industrialization was promoting significant population growth, as it was applied to many fields.', 'Parliament supported industrialization for its own interests.', 'Socioeconomic changes created divisions of labor that led to the development of self-conscious classes.']\",\"question\":\"This question refers to the following information.\\nRead the documents below.\\nSource 1\\nIf then by the Use of Machines, the Manufacture of Cotton, an Article which we import, and are supplied with from other Countries, and which can everywhere be procured on equal Terms, has met with such amazing Success, may not greater Advantages be reasonably expected from cultivating to the utmost the Manufacture of Wool, the Produce of our own Island, an Article in Demand in all Countries, almost the universal Clothing of Mankind?\\nIn the Manufacture of Woollens, the Scribbling Mill, the Spinning Frame, and the Fly Shuttle, have reduced manual Labour nearly One third, and each of them at its-first Introduction carried an Alarm to the Work People, yet each has contributed to advance the Wages and to increase the Trade, so that if an Attempt was now made to deprive us of the Use of them, there is no Doubt, but every Person engaged in the Business, would exert himself to defend them.\\n—Statement by the Cloth Merchants of Leeds, 1791\\nSource 2\\nCome, cropper lads of high renown,\\nWho love to drink good ale that's brown,\\nAnd strike each haughty tyrant down,\\nWith hatchet, pike, and gun!\\nOh, the cropper lads for me,\\nThe gallant lads for me,\\nWho with lusty stroke,\\nThe shear frames broke,\\nThe cropper lads for me!\\nWhat though the specials still advance,\\nAnd soldiers nightly round us prance;\\nThe cropper lads still lead the dance,\\nWith hatchet, pike, and gun!\\nOh, the cropper lads for me,\\nThe gallant lads for me,\\nWho with lusty stroke\\nThe shear frames broke,\\nThe cropper lads for me!\\n—Luddite Song, The Cropper's Song, c. 1812\\nSource 2 was motivated most by which of the following changes brought on by industrialization?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}