{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fdef6ead-d919-51ad-a270-5bb7ea097091","task_key":"all--test--fdef6ead-d919-51ad-a270-5bb7ea097091","task_revision_id":"1","upstream_id":"","short_description":"A suspected drug dealer was arrested after selling three grams of cocaine to…","config":"all","split":"test","body":"{\"choices\":\"['both indictments.', 'the state indictment, but not the federal indictment.', 'the federal indictment, but not the state indictment.', 'neither of the indictments.']\",\"question\":\"A suspected drug dealer was arrested after selling three grams of cocaine to undercover police agents. He was subsequently convicted in state court of possession with intent to distribute narcotics. Following his conviction, the drug dealer was sentenced to a prison term of 10 years. After being sentenced, the drug dealer's attorney appealed the conviction, citing jury misconduct. While his appeal was pending, the drug dealer was then indicted by both a federal and a state grand jury for conspiracy to distribute the same cocaine. The drug dealer's attorney filed motions to dismiss each of the new indictments on the grounds that they violate double jeopardy. In all likelihood, double jeopardy requires the dismissal of\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}