{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b18398a1-21e7-5a16-a187-d6fe4e22e2b5","task_key":"all--test--b18398a1-21e7-5a16-a187-d6fe4e22e2b5","task_revision_id":"1","upstream_id":"","short_description":"A new business contracted with a widget manufacturer for the purchase of 100,000…","config":"all","split":"test","body":"{\"choices\":\"['No, despite the expert testimony, anticipated profits for a new company are too speculative and are not awarded.', 'No, because the equipment failures were an excusable occurrence that the manufacturer could not foresee.', 'Yes, the lost profits damages were sufficiently proven by the evidence, including expert testimony.', 'Yes, a new company can usually collect its lost profits because it is easier to prove than if the company had many accounts to complicate its projections.']\",\"question\":\"A new business contracted with a widget manufacturer for the purchase of 100,000 widgets. The CEO of the new business told the widget maker's manager that a substantial profit hinged on receiving the widgets. The manufacturer discovered it could not produce the widgets due to equipment failures. The business sued the manufacturer for breach of contract, claiming lost profits of $10 million. The CEO testified to the company's projected sales in detail. A qualified expert supported his lost profit projections; there was no expert testimony by the manufacturer. Will the court allow the company to collect its claimed lost profit damages?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}