{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98e931dd-5827-5b9e-8786-7a86022b5300","task_key":"all--test--98e931dd-5827-5b9e-8786-7a86022b5300","task_revision_id":"1","upstream_id":"","short_description":"Needing a new supply of basketballs for the upcoming season, a basketball coach…","config":"all","split":"test","body":"{\"choices\":\"['Yes, provided both parties are viewed as being merchants.', 'Yes, provided that the three-day notice afforded the sporting goods store a reasonable time in which to perform.', \\\"No, because the coach's October 11 statement effectuated a waiver of any condition of timely delivery.\\\", 'No, because the sporting goods store, by its October 11 statement, did not promise to deliver the basketballs by October 20.']\",\"question\":\"Needing a new supply of basketballs for the upcoming season, a basketball coach placed an order with a sporting goods store to purchase 25 regulation basketballs. Since the season was scheduled to begin on November 1, the coach stated that he needed the basketballs delivered no later than October 15. The sporting goods store said it would deliver the basketballs by that date. On October 10, one of the sporting goods store's employees was moving items in the warehouse and accidentally destroyed a container of basketballs. As a result, the owner of the sporting goods store telephoned the coach on October 11 and said, \\\"We had an accident in our warehouse yesterday, and as a result we do not have enough basketballs to fill all of our orders. Therefore, I won't be able to deliver your basketballs on October 15. However, I have contacted our supplier to see if they can rush a delivery of basketballs to us. Although I can't promise it, I should be able to deliver the basketballs you ordered by October 20. \\\" The coach replied, \\\"No problem. I think I'll be able to get by until then. \\\" When the coach failed to receive the basketballs on October 20, he sent the following e-mail to the sporting goods store:\\\"I must have the basketballs no later than October 23. \\\" The e-mail was received and read the same day by the owner of the sporting goods store. If the sporting goods store fails to deliver the basketballs to the coach on October 23, will the coach be entitled to cancel the contract?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}