{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7057cebb-6d8d-51b1-9ecb-961c57497267","task_key":"all--test--7057cebb-6d8d-51b1-9ecb-961c57497267","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"[\\\"Napoleon's fear that if he did not continue to win military victories, he would lose power.\\\", \\\"Napoleon's belief that he must conquer all of Europe to conquer the press.\\\", \\\"Napoleon's confidence that he will be the greatest leader Europe has ever seen.\\\", \\\"Napoleon's appreciation of his soldiers and their collective efforts.\\\"]\",\"question\":\"This question refers to the following information.\\nThe following entries are in the diary of Napoleon Bonaparte.\\nParis has a short memory. If I remain longer doing nothing, I am lost. In this great Babylon one reputation quickly succeeds another. After I have been seen three times at the theatre, I shall not be looked at again. I shall therefore not go very frequently. (diary, 1798)\\nIf the press is not bridled, I shall not remain three days in power. (diary, 1799)\\nThe presence of a general is necessary; he is the head, he is the all in all of an army. It was not the Roman army that conquered Gaul, it was Caesar, it was not the Carthaginians that made the armies of the Roman republic tremble at the very gates of Rome, it was Hannibal. (diary, 1801)\\nMy power proceeds from my reputation, and my reputation from the victories I have won. My power would fall if I were not to support it with more glory and more victories. Conquest has made me what I am; only conquest can maintain me. (diary, 1802)\\nThe revolution in France is over and now there is only one party in France and I shall never allow the newspapers to say anything contrary to my interests. They may publish a few little articles with just a bit of poison in them, but one fine day I shall shut their mouths forever. (diary, 1805)\\nNapoleon's entry from 1802 indicates that his conquest of most of Europe was due in part to which of the following?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}