{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67be5900-fbb2-56d8-8833-a7e8ccdf4740","task_key":"all--test--67be5900-fbb2-56d8-8833-a7e8ccdf4740","task_revision_id":"1","upstream_id":"","short_description":"Which of the following generally results when the Senate and House of…","config":"all","split":"test","body":"{\"choices\":\"['The president signs the version he prefers.', \\\"The bill goes back to each house's committee and restarts the legislative process.\\\", 'All amendments to the bill are invalidated, and the original version of the bill is sent to the president to sign.', 'The two legislative bodies form a conference committee.']\",\"question\":\"Which of the following generally results when the Senate and House of Representatives pass different versions of the same bill?\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}