{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"405f7b0a-5d7b-5444-b2e9-cb92af07a05d","task_key":"all--test--405f7b0a-5d7b-5444-b2e9-cb92af07a05d","task_revision_id":"1","upstream_id":"","short_description":"A purchaser orally agreed with the seller to buy seller's house for $100,000.…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because there was an oral agreement and the check for the down payment was evidence of consideration being paid.', 'No, because the offer was not accepted \\\\x80\\\\x94 prior to an acceptance the seller had the right to sell to anyone.', 'Yes, because the buyer and seller had a meeting of the minds evidenced by their oral understanding, the check and the written offer.', 'No, because specific performance does not apply in a residential real estate transaction.']\",\"question\":\"A purchaser orally agreed with the seller to buy seller's house for $100,000. The purchaser gave the seller a written offer and a post-dated check for 5% of the price. Before the seller signed the offer form or deposited the check, he received an offer of his full asking price of $120,000 from another party. The seller called the purchaser and called the deal off. The purchaser sued for specific performance on the basis that the oral understanding, along with the written offer and the check, constituted an enforceable agreement. Will the purchaser likely prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}