{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad6382ed-9317-5a31-b22d-d61a3b4e6b4b","task_key":"all--test--ad6382ed-9317-5a31-b22d-d61a3b4e6b4b","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Medievalism', 'Empiricism', 'Rationalism', 'Romanticism']\",\"question\":\"This question refers to the following information.\\n\\\"But you, my dear Pangloss,\\\" said Candide, \\\"how can it be that I behold you again?\\\"\\n\\\"It is true,\\\" said Pangloss, \\\"that you saw me hanged&….A surgeon purchased my body, carried home, and dissected me. He began with making a crucial incision on me from the navel to the clavicula. One could not have been worse hanged than I was. The executioner of the Holy Inquisition was a sub-deacon, and knew how to burn people marvellously well, but he was not accustomed to hanging. The cord was wet and did not slip properly, and besides it was badly tied; in short, I still drew my breath, when the crucial incision made me give such a frightful scream that my surgeon fell flat upon his back&…[At length he] sewed up my wounds; his wife even nursed me. I was upon my legs at the end of fifteen days&….\\nOne day I took it into my head to step into a mosque, where I saw an old Iman and a very pretty young devotee who was saying her paternosters&….She dropped her bouquet; I picked it up, and presented it to her with a profound reverence. I was so long in delivering it that the Iman began to get angry, and seeing that I was a Christian he called out for help. They carried me before the cadi, who ordered me a hundred lashes on the soles of the feet and sent me to the galleys. I was chained to the very same galley and the same bench as the young Baron. On board this galley there were four young men from Marseilles, five Neapolitan priests, and two monks from Corfu, who told us similar adventures happened daily. The Baron maintained that he had suffered greater injustice than I&….We were continually disputing, and received twenty lashes with a bull's pizzle when the concatenation of universal events brought you to our galley, and you were good enough to ransom us.\\\"\\n\\\"Well, my dear Pangloss,\\\" said Candide to him, \\\"when you had been hanged, dissected, whipped, and were tugging at the oar, did you always think that everything happens for the best?\\\"\\n\\\"I am still of my first opinion,\\\" answered Pangloss, \\\"for I am a philosopher and I cannot retract, especially as Leibnitz could never be wrong; and besides, the pre-established harmony is the finest thing in the world, and so is his plenum and materia subtilis.\\\"\\nVoltaire, French Enlightenment writer, Candide, 1759\\nThe themes of the passage and the mode in which Pangloss tells them show the influence of\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}