{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c80d191-ea8a-525b-83ff-71506d6b8840","task_key":"all--test--6c80d191-ea8a-525b-83ff-71506d6b8840","task_revision_id":"1","upstream_id":"","short_description":"Congress recently passed a law that would grant, for free, federally owned buses…","config":"all","split":"test","body":"{\"choices\":\"['The gift violates the equal protection rights of secular organizations.', 'The gift violates the establishment clause.', 'The gift is a taking of federal property without just compensation.', 'The gift violates the commerce clause.']\",\"question\":\"Congress recently passed a law that would grant, for free, federally owned buses to a religious organization. The congressional statute stipulates that the religious organization must use the buses to travel across America to spread the moral message of sexual abstinence to teenagers at school assemblies. Which of the following is the strongest constitutional grounds for invalidating the gift of the buses to the religious organization?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}