{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c897ba0-fce7-544e-a3af-e85fb0341834","task_key":"all--test--3c897ba0-fce7-544e-a3af-e85fb0341834","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The differing views of Catholics and Reformers on the appropriate definition of \\\"Holy Ghost\\\"', 'The differing views of Catholics and Reformers on the appropriate use and interpretation of biblical texts', 'The differing views of Catholics and Reformers on the appropriate theological belief about the divinity of Jesus', 'The differing views of Catholics and Reformers on the appropriate method of baptism']\",\"question\":\"This question refers to the following information.\\n\\\"The Romanists have, with great adroitness, drawn three walls round themselves, with which they have hitherto protected themselves, so that no one could reform them, whereby all Christendom has fallen terribly.\\nFirstly, if pressed by the temporal power, they have affirmed and maintained that the temporal power has no jurisdiction over them, but, on the contrary, that the spiritual power is above the temporal.\\nSecondly, if it were proposed to admonish them with the Scriptures, they objected that no one may interpret the Scriptures but the Pope.\\nThirdly, if they are threatened with a council, they pretend that no one may call a council but the Pope…\\n…The second wall is even more tottering and weak: that they alone pretend to be considered masters of the Scriptures; although they learn nothing of them all their life. They assume authority, and juggle before us with impudent words, saying that the Pope cannot err in matters of faith, whether he be evil or good, albeit they cannot prove it by a single letter. That is why the canon law contains so many heretical and unchristian, nay unnatural, laws; but of these we need not speak now. For whereas they imagine the Holy Ghost never leaves them, however unlearned and wicked they may be, they grow bold enough to decree whatever they like. But were this true, where were the need and use of the Holy Scriptures? Let us burn them, and content ourselves with the unlearned gentlemen at Rome, in whom the Holy Ghost dwells, who, however, can dwell in pious souls only. If I had not read it, I could never have believed that the devil should have put forth such follies at Rome and find a following.\\\"\\nMartin Luther, Address to the Nobility of the German Nation, 1520\\nWhen the author of the passage above discusses the \\\"second wall,\\\" to what is he referring?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}