{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7adedf95-b982-599a-8067-6f058aea1ca4","task_key":"all--test--7adedf95-b982-599a-8067-6f058aea1ca4","task_revision_id":"1","upstream_id":"","short_description":"While ratifying the Constitution, the primary point of disagreement between…","config":"all","split":"test","body":"{\"choices\":\"['whether representation in Congress should be equal among states or proportional to population', 'the strength of the federal government relative to the states', 'the future of the slave trade', 'whether people had inalienable rights']\",\"question\":\"While ratifying the Constitution, the primary point of disagreement between Federalists and Anti-Federalists was over\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}