{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"07933b73-0dcd-5135-9a84-a2ca93b04a6d","task_key":"all--test--07933b73-0dcd-5135-9a84-a2ca93b04a6d","task_revision_id":"1","upstream_id":"","short_description":"On September 1, a manufacturer of portable drinking fountains mailed to a…","config":"all","split":"test","body":"{\"choices\":\"['valid, because the retailer had not changed its position in reliance on the September 1 offer.', 'valid, because there was no consideration to support an option contract.', 'not valid, because the retailer had 90 days in which to accept.', 'not valid, because the manufacturer gave assurance that the offer would remain open until October 1.']\",\"question\":\"On September 1, a manufacturer of portable drinking fountains mailed to a drinking fountain retailer a signed offer that stated: \\\"Have 200 drinking fountains available at $100 each for October delivery. Be advised that this offer will remain open until October 1. \\\"On September 30, the retailer mailed the following letter, which was received by the manufacturer on October 1: \\\"Your offer is hereby accepted, but request delivery of 100 drinking fountains in October and 100 in November. \\\"However, on September 29, the manufacturer sent a fax to the retailer revoking its offer, which was received the same day. This revocation is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}