{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2abe3f1-60e3-52c5-841d-dedfc528ff64","task_key":"all--test--a2abe3f1-60e3-52c5-841d-dedfc528ff64","task_revision_id":"1","upstream_id":"","short_description":"A manufacturer sued a buyer in federal court for failing to make timely payments…","config":"all","split":"test","body":"{\"choices\":\"['The judgment is clearly erroneous because it was based solely on documentary evidence.', 'The manufacturer was required to file proposed findings and conclusions before the trial court ruled.', 'The trial court erred because it announced the judgment without giving the parties an opportunity to submit proposed findings and conclusions.', 'The trial court erred by not providing findings and conclusions.']\",\"question\":\"A manufacturer sued a buyer in federal court for failing to make timely payments under the parties' sales contract. The case was tried to the court solely on documentary evidence. Immediately after the close of the evidence, the judge announced from the bench, \\\"Judgment shall be entered for the manufacturer,\\\" and judgment was so entered. The buyer has appealed the judgment. What is the buyer's best argument for persuading the appellate court to reverse the judgment?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}