{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20faf67f-573c-5d14-a26d-3d837671aeae","task_key":"all--test--20faf67f-573c-5d14-a26d-3d837671aeae","task_revision_id":"1","upstream_id":"","short_description":"At the close of all the evidence in a jury trial, Defendant moves for a directed…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because Defendant has raised new grounds.', 'Yes, because Defendant had ten days after the jury returned its verdict within which to move to set aside the verdict.', 'No, because the court denied the motion for directed verdict rather than reserving ruling.', 'No, because the court entered final judgment for Plaintiff before the motion to set aside the verdict was filed.']\",\"question\":\"At the close of all the evidence in a jury trial, Defendant moves for a directed verdict. After much argument, the court denies the motion. Subsequently, the jury returns a verdict for Plaintiff. The day after the jury returns its verdict, the court enters judgment for Plaintiff. One week later, Defendant moves to set aside the verdict and have judgment entered in accordance with its motion for directed verdict. In the motion, Defendant raises arguments that were not raised at trial. Plaintiff's counsel objects to the court even hearing the motion to set aside the verdict. Should the court consider the motion?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}