{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97b9f5d5-7749-5d16-8d2f-66d26fb82865","task_key":"all--test--97b9f5d5-7749-5d16-8d2f-66d26fb82865","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['revamped the economy of the new French Republic', 'successfully harnessed the human resources of the new French Republic', 'reformed the religious rituals of the Church in the new French Republic', 'brought about its own destruction']\",\"question\":\"This question refers to the following information.\\nFrom this moment until that in which the enemy shall have been driven from the soil of the Republic, all Frenchmen are in permanent requisition for the service of the armies. The young men shall go to battle; the married men shall forge arms and transport provisions; the women shall make tents and clothing and shall serve in the hospitals; the children shall turn old linen into lint; the aged shall betake themselves to the public places in order to arouse the courage of the warriors and preach the hatred of kings and the unity of the Republic. …\\nThe Committee of Public Safety is charged to take all necessary measures to set up without delay an extraordinary manufacture of arms of every sort which corresponds with the ardor and energy of the French people. It is, accordingly, authorized to form all the establishments, factories, workshops, and mills which shall be deemed necessary for the carrying on of these works, as well as to put in requisition, within the entire extent of the Republic, the artists and workingmen who can contribute to their success.\\nThe representatives of the people sent out for the execution of the present law shall have the same authority in their respective districts, acting in concert with the Committee of Public Safety; they are invested with the unlimited powers assigned to the representatives of the people to the armies.\\nThe Levée en Masse, August 23, 1793\\nThe passage can be used as one example of the way in which the Committee of Public Safety\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}