{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be628150-e85c-5332-b73e-2533ce1d4622","task_key":"all--test--be628150-e85c-5332-b73e-2533ce1d4622","task_revision_id":"1","upstream_id":"","short_description":"A state has recently enacted a statute wherein aliens are forbidden from owning…","config":"all","split":"test","body":"{\"choices\":\"['the statute violates the privileges or immunities clause of the Fourteenth Amendment.', 'the statute violates the contract clause.', 'the statute violates the commerce clause in that it interferes with land ownership.', 'the statute violates the equal protection clause of the Fourteenth Amendment.']\",\"question\":\"A state has recently enacted a statute wherein aliens are forbidden from owning more than 10 acres of land within the state. Subsequent to the statute a resident alien enters into a contract to buy 50 acres of land located in the state. Assume that the statute empowers the state to bring an ejectment action against any alien who owns more than 10 acres of land. If the resident alien brings an action in federal court to enjoin the state from enforcing the statute against him, his best argument is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}