{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ccaed17a-040d-5733-a118-ffd70c51b767","task_key":"all--test--ccaed17a-040d-5733-a118-ffd70c51b767","task_revision_id":"1","upstream_id":"","short_description":"Which of the following statements is true of congressional incumbents who run…","config":"all","split":"test","body":"{\"choices\":\"['Incumbent senators are more likely to be reelected than are incumbent members of the House of Representatives.', 'Incumbents are prohibited by law from spending more on their reelection campaigns than their challengers spend.', 'Incumbents have a great advantage over challengers because they are better known and can raise campaign funds more easily.', 'Ever since the 1994 election, the majority of congressional incumbents have failed in their reelection attempts.']\",\"question\":\"Which of the following statements is true of congressional incumbents who run for reelection?\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}