{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac2517a1-50cb-5e9c-9350-d7416a0282c0","task_key":"all--test--ac2517a1-50cb-5e9c-9350-d7416a0282c0","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['American desires for independence from Great Britain', 'American rejection of Enlightenment ideals', 'A long tradition of self-rule in the American colonies', 'The weakening of economic ties between America and Great Britain']\",\"question\":\"This question refers to the following information.\\nIt is natural, it is a privilege, I will go farther, it is a right, which all free men claim, that they are entitled to complain when they are hurt. They have a right publicly to remonstrate against the abuses of power in the strongest terms, to put their neighbors upon their guard against the craft or open violence of men in authority, and to assert with courage the sense they have of the blessings of liberty, the value they put upon it, and their resolution at all hazards to preserve it as one of the greatest blessings heaven can bestow.… But to conclude: The question before the Court and you, Gentlemen of the jury, is not of small or private concern. It is not the cause of one poor printer, nor of New York alone, which you are now trying. No! It may in its consequence affect every free man that lives under a British government on the main of America. It is the best cause. It is the cause of liberty. And I make no doubt but your upright conduct this day will not only entitle you to the love and esteem of your fellow citizens, but every man who prefers freedom to a life of slavery will bless and honor you as men who have baffled the attempt of tyranny, and by an impartial and uncorrupt verdict have laid a noble foundation for securing to ourselves, our posterity, and our neighbors, that to which nature and the laws of our country have given us a right to liberty of both exposing and opposing arbitrary power (in these parts of the world at least) by speaking and writing truth.\\n—Andrew Hamilton, concluding argument, libel trial of newspaper editor\\nJohn Peter Zenger, August 4, 1735\\nHamilton's success in the Zenger case most directly reflects which of the following?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}