{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"755f37f3-1fd1-5ea2-9ee1-9e40ac71e690","task_key":"default--test--1470","task_revision_id":"1","upstream_id":"1470","short_description":"A seller contracted to manufacture 1,000 toasters for a buyer for a specified…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Yes, because even though the seller breached the contract, there are no damages since the competitor's toasters are of equal quality to the seller's toasters.\\\", 'No, because the seller assigned a right despite the contractual prohibition.', 'Yes, because the quality of the toasters did not change even though the manufacturer did.', 'No, because the seller violated the contractual prohibition by delegating its duty to a competitor.', 'No, because \\\"this contract may not be assigned\\\" means that duties may not be delegated, and the seller delegated a duty.', 'No, because the non-assignment clause voids the contract when violated.', 'No, because the competitor is not a party to the original contract.', 'Yes, because the seller informed the buyer about the change in manufacturer.', 'Yes, because the non-assignment provision is not enforceable since public policy favors free assignment and delegation.', 'Yes, because the buyer did not explicitly reject the toasters manufactured by the competitor.']\",\"question\":\"A seller contracted to manufacture 1,000 toasters for a buyer for a specified price. The contract contained a provision that clearly stated: \\\"This contract may not be assigned, and any violation of this prohibition voids the contract.\\\" After the contract was signed, the seller informed the buyer that the toasters would be manufactured by a competitor of the seller. Citing the non-assignment provision, the buyer claimed that it was no longer bound by the contract. Toasters manufactured by the competitor were of equal quality to toasters manufactured by the seller. Is the buyer bound by the contract?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}