{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c208888c-8920-52f3-b0f0-e7254f98d1c1","task_key":"all--test--c208888c-8920-52f3-b0f0-e7254f98d1c1","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The advantage of superior numbers', 'The superior military skill of mercenaries', 'Taking advantage of divisions among the Indians', 'Effective use of European artillery']\",\"question\":\"This question refers to the following information.\\nOf all the band of adventurous cavaliers, whom Spain, in the sixteenth century, sent forth on the career of discovery and conquest, there was none more deeply filled with the spirit of romantic enterprise than Hernando Cortes. Dangers and difficulties, instead of deterring, seemed to have a charm in his eyes.… He conceived, at the first moment of his landing in Mexico, the design of its conquest. When he saw the strength of its civilization, he was not turned from his purpose.… This spirit of knight-errantry might lead us to undervalue his talents as a general, and to regard him merely in the light of a lucky adventurer. But this would be doing him injustice; for Cortes was certainly a great general, if that man be one, who performs great achievements with the resources which his own genius has created. There is probably no instance in history, where so vast an enterprise has been achieved by means apparently so inadequate.… He brought together the most miscellaneous collection of mercenaries who ever fought under one standard: adventurers from Cuba and the Isles, craving for gold; hidalgos, who came from the old country to win laurels; … wild tribes of the natives from all parts of the country, who had been sworn enemies from their cradles, and who had met only to cut one another's throats, and to procure victims for sacrifice; men, in short, differing in race, in language, and in interests, with scarcely anything in common among them. Yet this motley congregation was assembled in one camp, compelled to bend to the will of one man, to consort together in harmony, to breathe, as it were, one spirit, and to move on a common principle of action!\\n—William Hickling Prescott, History of the Conquest of Mexico, 1843\\nAs Prescott makes clear in the passage above, an important reason for Cortes's military success was which of the following?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}