{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a6cf52a-4c1f-52e4-9aa3-09c556d1c065","task_key":"default--test--1256","task_revision_id":"1","upstream_id":"1256","short_description":"A manufacturer sued a buyer in federal court for failing to make timely payments…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The buyer was not given sufficient time to prepare a defense.', 'The trial court erred by making a decision without hearing oral arguments.', 'The manufacturer was required to file proposed findings and conclusions before the trial court ruled.', \\\"The trial court erred because it did not consider the buyer's financial circumstances.\\\", 'The trial court erred by not providing findings and conclusions.', 'The judgment is clearly erroneous because it was based solely on documentary evidence.', 'The trial court erred by not allowing the buyer to cross-examine the manufacturer.', 'The judgment is invalid because the trial court did not consider the credibility of the documentary evidence.', 'The manufacturer failed to establish a prima facie case against the buyer.']\",\"question\":\"A manufacturer sued a buyer in federal court for failing to make timely payments under the parties' sales contract. The case was tried to the court solely on documentary evidence. Immediately after the close of the evidence, the judge announced from the bench, \\\"Judgment shall be entered for the manufacturer,\\\" and judgment was so entered. The buyer has appealed the judgment. What is the buyer's best argument for persuading the appellate court to reverse the judgment?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}