{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3466c424-83fe-5f07-ab89-1a0d587f718d","task_key":"default--test--1707","task_revision_id":"1","upstream_id":"1707","short_description":"A woman took her car to a mechanic to be serviced. After doing a thorough…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The mechanic had a legal obligation to replace the worn belt in order to ensure the car's safety.\\\", 'The parties entered into a collateral oral agreement for the replacement of the worn belt that was excludable from the integrated writing.', 'The written contract was not specific about what services were included in the $1,000 fee.', 'The mechanic performed an action beyond the agreed contract and hence has the right to additional compensation.', 'An implied-in-law contract was created at the moment the mechanic replaced the worn belt.', 'The woman verbally agreed to the extra work and therefore should cover the additional costs.', 'The oral agreement was not legally binding as it was not included in the written contract.', \\\"The writing was construed as a complete integration of the parties' agreement.\\\", \\\"The writing was construed as a partial integration of the parties' agreement.\\\", 'The mechanic could reasonably assume that the woman would want the worn belt replaced.']\",\"question\":\"A woman took her car to a mechanic to be serviced. After doing a thorough inspection of the car, the mechanic told the woman that her car needed extensive work and that he could do all that was needed for $1,000. The parties then entered into a written contract to have the car serviced for $1,000. As the woman remembers it, at the time the written contract was formed, the parties orally agreed that if the car required any additional servicing above that detailed in the contract, it would be included in the $1,000 fee. The mechanic, however, has no recollection of any such oral agreement. While servicing the car, the mechanic discovered that a belt was severel' worn and need to be replaced. The belt replacement was not listed as work to be performed in the written contract. The mechanic telephoned the woman and told her he had found the worn belt and felt he should replace it. The woman told him to do so. When the mechanic was finished, he presented the woman with a bill for $1,100, representing the $1,000 called for by the contract and $100 for the replacement of the worn belt, which is a reasonable amount. The woman refused to pay any more than$1,000. The mechanic sued the woman to collect the $100 for the replacement of the worn belt. If the court rules in favor of the mechanic, which of the following is the most likely cause for such a ruling?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}