{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db339224-746d-585b-a46d-2e9b299b21a6","task_key":"all--test--db339224-746d-585b-a46d-2e9b299b21a6","task_revision_id":"1","upstream_id":"","short_description":"A resident announced his candidacy for state representative. A law in the state…","config":"all","split":"test","body":"{\"choices\":\"['hold the authenticating requirement unconstitutional as violative of the due process clause of the Fourteenth Amendment.', 'hold the authenticating requirement unconstitutional as violative of the equal protection clause of the Fourteenth Amendment.', 'dismiss the cause of action because state election procedures are a sovereign state function.', 'assign the burden of proving the validity of the authenticating requirement to thestate.']\",\"question\":\"A resident announced his candidacy for state representative. A law in the state requires new political entrants, regardless of party affiliation, to obtain five times the number of signatures as other candidates who have run for office previously. The resident, however, failed to obtain the necessary number of authenticating signatures to have his name placed on the ballot. On the substantive constitutional issue regarding the validity of the authenticating requirement, the most probable judicial resolution will be to\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}