{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e27374e6-4a35-5f68-9f62-cd61e490fcf9","task_key":"default--test--1443","task_revision_id":"1","upstream_id":"1443","short_description":"On June 1, an appliance manufacturer telephoned a supplier to determine whether…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Yes, because the manufacturer had not received any orders for the machines with the specified motors at the time of the supplier's refusal.\\\", \\\"No, because the manufacturer's representative verbally agreed to the deal over the phone.\\\", 'No, because the supplier failed to object to the contents of the note sent by the manufacturer.', 'Yes, because the supplier was no longer carrying the model of the motor at the time of the call.', 'Yes, because there is no writing that contains the quantity term of the contract.', \\\"Yes, because the manufacturer's note failed to contain a signature.\\\", \\\"No, because the manufacturer's representative sent the note on company letterhead.\\\", 'No, because the supplier did not inform the manufacturer about not being able to supply the motors in time.', 'Yes, because the supplier never agreed to the contract in writing.', 'No, because the manufacturer distributed the catalogs in reliance on the contract, making the contract enforceable under the statute of frauds.']\",\"question\":\"On June 1, an appliance manufacturer telephoned a supplier to determine whether the supplier could provide 300 washing machine motors of a particular model by October 1. The supplier offered to do so at a price of $300 per motor (a total price of $90,000). The manufacturer's representative said, \\\"Deal.\\\" The next day the manufacturer's representative sent the supplier an unsigned note on company letterhead that stated, \\\"I am happy that you are going to supply us with the motors. I will call you soon to talk about another order.\\\" The manufacturer then sent catalogs to its regular customers advertising washing machines that included the specified motors. The manufacturer did not hear from the supplier until July 1, when the supplier called to say that it would be unable to supply the motors because it was no longer carrying that model. At that time, the manufacturer had received no orders for the machines with the specified motors. The manufacturer sued the supplier for breach of contract, and the supplier raised the statute of frauds as a defense. Is the supplier's statute of frauds defense likely to succeed?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}