{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b76ff6a9-6581-5381-8357-7599b7db5444","task_key":"all--test--b76ff6a9-6581-5381-8357-7599b7db5444","task_revision_id":"1","upstream_id":"","short_description":"An auto retailer started a marketing promotion on a new hybrid model sedan. It…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because this is a classic bilateral contract, making it necessary that the company live up to its set of promises made to her.', 'No, this was merely puffing, was not an offer to contract with any particular person and was without consideration.', 'Yes, this was a unilateral contract in which the offeree acted upon the offer by performing the terms of the offer, thus creating a binding contract.', 'No, because she did not formally notify the company that she was accepting the offer before she started performing.']\",\"question\":\"An auto retailer started a marketing promotion on a new hybrid model sedan. It advertised that anyone who purchased the car in 2013 and did not get at least 50 miles per gallon average gas mileage during the first 60 days, would receive a payment of $10,000 cash from the company. Record-keeping and inspection procedures were required to assure accurate reporting by the vehicle owner. The promotion was widely publicized. A female customer purchased a hybrid model during the promotion period, followed all the rules, and recorded only 42 mpg in the first 60 days. She demanded the rebate but the company stalled for months and then stopped responding to her inquiries. Does she have a contractual right to collect the $10,000?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}