{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"567ebd32-0ee8-5d52-98aa-590ecf350f74","task_key":"all--test--567ebd32-0ee8-5d52-98aa-590ecf350f74","task_revision_id":"1","upstream_id":"","short_description":"A defendant was prosecuted for mail fraud. At trial, the defendant moved to have…","config":"all","split":"test","body":"{\"choices\":\"['No, provided that the government can show that the agentÕs presence is essential to the presentation of its case.', 'No, because the government has a right to have its designated representative remain in the courtroom through-out the trial.', 'Yes, because the agentÕs testimony might be influenced by the testimony of other witnesses.', 'Yes, because the defendant has a right to exclude all persons who may be called as government witnesses.']\",\"question\":\"A defendant was prosecuted for mail fraud. At trial, the defendant moved to have all witnesses excluded from the courtroom, and the court granted the motion. The government named the investigating FBI agent as its designated representative. Upon learning that the agent would be giving testimony during the trial, the defendant moved that the agent also be excluded from the courtroom. Should the defendantÕs motion be granted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}