{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3121bb2-87c8-57d8-81c1-22adf1730e2f","task_key":"default--test--1331","task_revision_id":"1","upstream_id":"1331","short_description":"At the close of all the evidence in a jury trial, Defendant moves for a directed…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the court entered final judgment for Plaintiff before the motion to set aside the verdict was filed.', 'Yes, because the court may reconsider its decision on the motion for directed verdict.', 'Yes, because Defendant had ten days after the jury returned its verdict within which to move to set aside the verdict.', \\\"Yes, because Defendant has the right to appeal the jury's verdict.\\\", 'No, because the motion to set aside the verdict is not timely.', 'No, because the court denied the motion for directed verdict rather than reserving ruling.', 'Yes, because Defendant has raised new grounds.', 'Yes, because the court has the discretion to consider any motion that is relevant to the case.', \\\"No, because Plaintiff's counsel objects to the motion to set aside the verdict.\\\"]\",\"question\":\"At the close of all the evidence in a jury trial, Defendant moves for a directed verdict. After much argument, the court denies the motion. Subsequently, the jury returns a verdict for Plaintiff. The day after the jury returns its verdict, the court enters judgment for Plaintiff. One week later, Defendant moves to set aside the verdict and have judgment entered in accordance with its motion for directed verdict. In the motion, Defendant raises arguments that were not raised at trial. Plaintiff's counsel objects to the court even hearing the motion to set aside the verdict. Should the court consider the motion?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}