{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"940fc089-ee16-5c8e-a5f6-e2f305cf6a1a","task_key":"all--test--940fc089-ee16-5c8e-a5f6-e2f305cf6a1a","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Calvinists believed that the state is absolutely subordinate to the religious organization.', 'Calvinists believed in consubstantiation while Lutherans believed in transubstantiation.', 'Lutherans used the reformed faith to their political advantage to break free from domination from the pope and the Holy Roman Empire.', 'Lutherans insisted upon maintaining all of the seven sacraments while the Calvinists chose to honor only two of them.']\",\"question\":\"This question refers to the following information.\\nRead the excerpts below.\\nThis corruption is repeatedly designated by Paul by the term sin . . . such as adultery, fornication, theft, hatred, murder, revellings, he terms, in the same way, the fruits of sin, though in various passages of Scripture . . . we are, merely on account of such corruption, deservedly condemned by God, to whom nothing is acceptable but righteousness, innocence, and purity.\\n—John Calvin, from The Institutes of Christian Religion, Book 2: Chapter 1, 1545\\nThe covenant of life is not preached equally to all, and among those to whom it is preached, does not always meet with the same reception. This diversity displays the unsearchable depth of the divine judgment, and is without doubt subordinate to God's purpose of eternal election. But if it is plainly owing to the mere pleasure of God that salvation is spontaneously offered to some, while others have no access to it, great and difficult questions immediately arise, questions which are inexplicable, when just views are not entertained concerning election and predestination[,] . . . the grace of God being illustrated by the contrast, viz., that he does not adopt all promiscuously to the hope of salvation, but gives to some what he denies to others.\\n—John Calvin, from The Institutes of Christian Religion, Book 3: Chapter 21, 1545\\nWhich of the following is the biggest contrast between the ideas of Calvinists when compared to their fellow Protestant Lutherans?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}