{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7743ac7d-0aa5-50a9-8ed6-0f6f8e8ae34d","task_key":"default--test--1422","task_revision_id":"1","upstream_id":"1422","short_description":"A company offered to sell several loads of landscaping stones at a specified…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The offeree will prevail because verbal rejection of the contract is considered valid.', \\\"The offeror will prevail because the offeree's modifications to the terms were not significant enough to void the contract.\\\", 'The offeror will prevail because it shipped the goods when it received the first acceptance.', 'The offeree will prevail because it returned the goods, effectively cancelling the contract.', \\\"The offeree will prevail because it rejected the offer when it called the offeror's manager.\\\", 'The offeror will prevail because the offeree did not officially cancel the contract in writing.', 'The offeror will prevail because the posting of the acceptance finalized a contract between the parties.', 'The offeree will prevail because the modifications to the terms were not accepted by the offeror.', 'The offeror will prevail because the offeree already accepted the terms when it sent the initial notification.', 'The offeree will prevail because it never accepted the offer on the same terms specified.']\",\"question\":\"A company offered to sell several loads of landscaping stones at a specified price and itemized terms. The offeree sent a notification to the offeror stating, \\\"We have received your offer to sell landscaping stone per specified terms this date.\\\" The offeror shipped the goods upon receiving that notification. On the date of shipment, the offeree posted an acceptance form with slight modifications to the terms. After the goods arrived, the offeree called the offeror's manager and told him that the company was unable to accept the offer at this time, he was shipping the goods back, and to \\\"ignore any paperwork we sent.\\\" The offeror sued the offeree for breach of contract. Who will prevail?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}