{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c405c009-e32e-5fa3-aed4-6b5501df8b61","task_key":"default--test--1948","task_revision_id":"1","upstream_id":"1948","short_description":"A retailer brought a federal diversity action against a wholesaler, alleging…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, because the wholesaler's challenge is merely a disagreement with the jury's verdict, not a legal issue for the appellate court.\\\", \\\"Yes, because the wholesaler's challenge pertains to a fundamental aspect of the trial process.\\\", \\\"Yes, because the wholesaler has the right to appeal the trial court's decisions.\\\", 'No, because the wholesaler did not raise the sufficiency-of-the-evidence issue in a motion for judgment as a matter of law before the case went to the jury.', 'Yes, because the challenge was raised and ruled on by the trial court before the wholesaler filed the appeal.', 'Yes, because the appellate court has the power to review all aspects of the case, including the sufficiency of the evidence.', \\\"No, because a determination of the sufficiency of the evidence is solely within sufficiency of the evidence is solely within the jury's province.\\\", 'Yes, because, as the trial court acknowledged, the wholesaler has strong arguments on the challenge.']\",\"question\":\"A retailer brought a federal diversity action against a wholesaler, alleging breach of contract and fraudulent misrepresentation. After the parties presented their evidence at trial, the court instructed the jury on the law. Neither party filed a motion for judgment as a matter of law before the case went to the jury. The jury found for the retailer on both claims. After the court entered judgment on the verdict, the wholesaler moved for a new trial and for judgment as a matter of law, arguing that the evidence was insufficient to support the jury verdict on either claim. The court acknowledged that there had been problems with some of the evidence, but it denied the motions. The wholesaler appealed, challenging the sufficiency of the evidence. Should the appellate court consider the wholesaler's challenge?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}