{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48b991d4-f8f2-5c8b-b968-13f1d8a3a120","task_key":"all--test--48b991d4-f8f2-5c8b-b968-13f1d8a3a120","task_revision_id":"1","upstream_id":"","short_description":"The origination clause of the Constitution states that","config":"all","split":"test","body":"{\"choices\":\"['the Bill of Rights must not be infringed upon by any state legislature', 'Congress and the president must jointly create the annual federal budget', 'revenue bills must be initiated in the House of Representatives', 'the Supreme Court has original jurisdiction over all constitutional matters']\",\"question\":\"The origination clause of the Constitution states that\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}