{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd740b3d-1989-5bea-8eef-885a27ecda44","task_key":"all--test--cd740b3d-1989-5bea-8eef-885a27ecda44","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['created a spoils system that bloated the size of the bureaucracy', 'laid the foundations of the welfare state', 'called for an ambitious program of road building and public works', 'vigorously exercised federal power in foreign affairs']\",\"question\":\"This question refers to the following information.\\nThe man who mounted the steps of the Capitol, March 4, 1801, to claim the place of an equal between Pitt and Bonaparte, possessed a character which showed itself in acts; but person and manner can be known only by contemporaries, and the liveliest description was worth less than a moment of personal contact. Jefferson was very tall, six feet two-and-a-half inches in height; sandy-complexioned; shy in manner, seeming cold; awkward in attitude, and with little in his bearing that suggested command.… His skin was thin, peeling from his face on exposure to the sun, giving it a tettered appearance. This sandy face, with hazel eyes and sunny aspect; this loose, shackling person; this rambling and often brilliant conversation, belonged to the controlling influences of American history, more necessary to the story than three-fourths of the official papers, which only hid the truth. Jefferson's personality during these eight years appeared to be the government, and impressed itself, like that of Bonaparte, although by a different process, on the mind of the nation. In the village simplicity of Washington, he was more than a king, for he was alone in social as well as in political pre-eminence. Except the British Legation, no house in Washington was open to general society; the whole mass of politicians, even the Federalists, were dependent on Jefferson and \\\"The Palace\\\" for amusement; and if they refused to go there, they \\\"lived like bears, brutalized and stupefied.\\\"\\n—Henry Adams, History of the United States During the Administration of Thomas Jefferson, 1889\\nAlthough Jefferson believed that government should be small and limited, once in office he\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}