{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"881d26df-590d-5963-a054-9582fee6095e","task_key":"default--test--1436","task_revision_id":"1","upstream_id":"1436","short_description":"Over the years, a coin collector had amassed a large and valuable collection. On…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the retailer should have responded to the initial letter before the collection was sold to the novice collector.', \\\"No, because the collector had already sold the collection before the retailer's acceptance, making it invalid.\\\", \\\"No, because the retailer's acceptance was not valid as the collection had already been sold.\\\", \\\"Yes, because the retailer's knowledge of the sale to the novice collector was not confirmed until after he sent his acceptance e-mail.\\\", \\\"Yes, because the retailer's acceptance was made within the specified timeframe.\\\", \\\"No, because the retailer's reading the article on the internet on May 12 terminated the retailer's power of acceptance.\\\", 'Yes, because consideration is not necessary under the UCC for the creation of a sale of goods contract.', \\\"Yes, because the collector's May 11 letter did not effectuate a revocation, since it was not received by the retailer until May 13.\\\", \\\"Yes, because the retailer's e-mail was sent before receiving the collector's May 11 letter.\\\"]\",\"question\":\"Over the years, a coin collector had amassed a large and valuable collection. On May 1, the collector sent the following letter to a retailer who had long coveted the collector's coin collection:\\\"I've decided to part with my coin collection. Since you've always been interested in purchasing my collection for your store, I'll sell you my entire collection for $50,000. You have until May 15 to make up your mind. \\\"The retailer received this letter on May 3. On May 10, without notif'ing the retailer, the collector sold her coin collection to a novice collector, for $60,000. The next day, the collector sent a letter to the retailer, regretfully informing him that she had sold the coin collection. This letter was received by the retailer on May 13. However, on May 12, the retailer read an article on the internet, published by a reputable site that reports coin collecting news, concerning the collector's sale of her coin collection to the novice collector. After reading the article, the retailer immediately sent an e-mail to the collector accepting her offer to buy the coin collection. This e-mail was read by the collector that same afternoon, May 12. In all likelihood was the retailer's e-mail to the collector on May 12 effective as an acceptance?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}