{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b35162f-18d4-57d1-815c-906681b2f0f1","task_key":"all--test--4b35162f-18d4-57d1-815c-906681b2f0f1","task_revision_id":"1","upstream_id":"","short_description":"All of the following statements concerning the likelihood that a person will…","config":"all","split":"test","body":"{\"choices\":\"['When there is a strong front-runner in a state, people in that state are less likely to vote.', 'White-collar workers are more likely to vote than are blue-collar workers, with the exception of blue-collar workers who belong to unions.', 'Voters who are registered as independent are less likely to vote than those who are registered Democrats or Republicans.', 'There is no difference in the likelihood of voting among those with undergraduate degrees and those with postgraduate degrees.']\",\"question\":\"All of the following statements concerning the likelihood that a person will vote are true EXCEPT:\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}