{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80bc02df-0c6d-5e33-9379-63eed58b5f51","task_key":"all--test--80bc02df-0c6d-5e33-9379-63eed58b5f51","task_revision_id":"1","upstream_id":"","short_description":"Susan Seller mailed a letter to Bobby Buyer after Bobby called her inquiring if…","config":"all","split":"test","body":"{\"choices\":\"['For Bobby, since her acceptance was timely under the mail box rule.', 'For Susan, because her letter to Bobby was not an offer.', 'For Susan, because the sale to Harriett was made before Susan was aware that Bobby had accepted.', 'For Bobby, because her communication met the full price which Susan was asking for the automobile.']\",\"question\":\"Susan Seller mailed a letter to Bobby Buyer after Bobby called her inquiring if her antique 1932 Ford Roadster was for sale. The letter stated, \\\"I want to sell my antique Ford Roadster for a price above $20,000. If you are interested please contact me.\\\" Three days later Bobby sent a letter to Susan stating \\\"I agree to pay your asking price of $20,000 for the 1932 Ford Roadster.\\\" The next day Susan sold the car to Harriet Higherprice for $25,000. If Bobby brings suit against Susan for breach of contract, the likely outcome is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}