{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"91f67fdd-afae-512e-a5a7-11cb7b6d21b0","task_key":"default--test--1892","task_revision_id":"1","upstream_id":"1892","short_description":"An elderly woman underwent major surgery and spent two weeks in the hospital.…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, because the contract was voidable due to the woman's apparent incapacity.\\\", 'No, because the woman was under the influence of medication at the time of the agreement.', \\\"Yes, because the neighbor's reliance on the otherwise voidable contract made it enforceable.\\\", 'Yes, because the son has no authority to change the terms of the agreement.', \\\"Yes, because the woman's offer and the neighbor's acceptance created an enforceable contract.\\\", 'Yes, because the woman had the mental capacity to make a decision.', 'No, because the woman put nothing in writing.', \\\"No, because the neighbor took advantage of the woman's condition.\\\", 'No, because there was no legal documentation of the agreement.', 'Yes, because the neighbor had already agreed to the initial price.']\",\"question\":\"An elderly woman underwent major surgery and spent two weeks in the hospital. The woman continued to take powerful pain medication for several weeks after she returned home. During her recovery, she offered to sell her car for $450 to her neighbor, who owned a house-cleaning service. The neighbor said, \\\"That's great! I need a car to transport all the people who work for me to their job sites.\\\" In fact, the woman's car was worth $3,000, and the neighbor knew this. He was also aware that the woman had undergone surgery and noted that she seemed \\\"out of it\\\" because of the medication she was taking. Several days later, the woman's son found out about the deal and contacted the neighbor, telling him that the woman would sell him the car, but for $3,450. The next day, when the neighbor tendered $450 and demanded that the woman give him the car, she refused. If the neighbor sues the woman for breach of contract, will he be likely to prevail?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}