{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3dbbab87-8cf0-56d3-9a3d-7d604f4d8c12","task_key":"all--test--3dbbab87-8cf0-56d3-9a3d-7d604f4d8c12","task_revision_id":"1","upstream_id":"","short_description":"On Monday, a man told a gardener, \"I am having a party on Sunday and I want my…","config":"all","split":"test","body":"{\"choices\":\"['The gardener, because he never accepted the offer made by the man.', 'The gardener, because he offered to cure the defective performance by finishing the job on Monday morning.', \\\"The man, because the gardener's part performance necessarily implied an acceptance and a promise that he would render complete performance.\\\", \\\"The man, because under the doctrine of equitable estoppel, the gardener's part performance was evidence of his intent to honor the entire contract.\\\"]\",\"question\":\"On Monday, a man told a gardener, \\\"I am having a party on Sunday and I want my house to look good. If you will promise to mow my lawn by Saturday, I will pay you $50. \\\" On Friday, the gardener arrived at the man's home just as the man was leaving for work and began to mow the man's lawn. The man said nothing to the gardener but drove off as he saw the gardener unloading his lawn mower. When the man arrived home from work that evening, he noticed that only half of his lawn had been mowed. He then found a note from the gardener slipped into his mailbox. The note said:\\\"Sorry, but I ran out of gas to power the lawn mower and did not have time to buy more gas to finish the job. I'm taking the weekend off, but I will be back Monday morning to finish the job. \\\"If the man brings suit against the gardener for breach of contract, who is likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}