{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c5c2fa0-e22a-561f-a78d-9d33caf3fa62","task_key":"all--test--4c5c2fa0-e22a-561f-a78d-9d33caf3fa62","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['that the petitioners be released from slavery because slavery was incompatible with the Massachusetts constitution.', 'that the Massachusetts legislature extend reparations to the petitioners as compensation for their time in slavery.', 'that the petitioners receive land that had been expropriated from loyalists in order to reward them for their service to the Continental Army during the American Revolution.', 'that the Massachusetts legislature either grant the petitioners the right to vote or that it excuse them from paying taxes.']\",\"question\":\"This question refers to the following information.\\n\\\"The petition of several poor negroes and mulattoes, who are inhabitants of the town of Dartmouth, humbly showeth,—\\n\\\"That we being chiefly of the African extract, and by reason of long bondage and hard slavery, we have been deprived of enjoying the profits of our labor or the advantage of inheriting estates from our parents, as our neighbors the white people do, having some of us not long enjoyed our own freedom; yet of late, contrary to the invariable custom and practice of the country, we have been, and now are, taxed both in our polls and that small pittance of estate which, through much hard labor and industry, we have got together to sustain ourselves and families withall. We apprehend it, therefore, to be hard usage, and will doubtless (if continued) reduce us to a state of beggary, whereby we shall become a burthen to others, if not timely prevented by the interposition of your justice and your power.\\n\\\"Your petitioners further show, that we apprehend ourselves to be aggrieved, in that, while we are not allowed the privilege of freemen of the State, having no vote or influence in the election of those that tax us, yet many of our colour (as is well known) have cheerfully entered the field of battle in the defence of the common cause, and that (as we conceive) against a similar exertion of power (in regard to taxation), too well known to need a recital in this place.\\\"\\n—Paul Cuffe's Petition, Massachusetts, 1780\\nThe main purpose of the petition by Paul Cuffe, excerpted above, was to demand\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}