{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a73c29e1-362a-56a4-bead-7f462a15febe","task_key":"all--test--a73c29e1-362a-56a4-bead-7f462a15febe","task_revision_id":"1","upstream_id":"","short_description":"An elderly woman underwent major surgery and spent two weeks in the hospital.…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the contract was voidable due to the woman's apparent incapacity.\\\", 'No, because the woman put nothing in writing.', \\\"Yes, because the neighbor's reliance on the otherwise voidable contract made it enforceable.\\\", \\\"Yes, because the woman's offer and the neighbor's acceptance created an enforceable contract.\\\"]\",\"question\":\"An elderly woman underwent major surgery and spent two weeks in the hospital. The woman continued to take powerful pain medication for several weeks after she returned home. During her recovery, she offered to sell her car for $450 to her neighbor, who owned a house-cleaning service. The neighbor said, \\\"That's great! I need a car to transport all the people who work for me to their job sites.\\\" In fact, the woman's car was worth $3,000, and the neighbor knew this. He was also aware that the woman had undergone surgery and noted that she seemed \\\"out of it\\\" because of the medication she was taking. Several days later, the woman's son found out about the deal and contacted the neighbor, telling him that the woman would sell him the car, but for $3,450. The next day, when the neighbor tendered $450 and demanded that the woman give him the car, she refused. If the neighbor sues the woman for breach of contract, will he be likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}