{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5eea713f-1e49-5ff8-8cd4-d6e4fe201f2f","task_key":"all--test--5eea713f-1e49-5ff8-8cd4-d6e4fe201f2f","task_revision_id":"1","upstream_id":"","short_description":"In June a computer retailer ordered 100 computers, with various configurations,…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because there was only a minimal difference between the offer and acceptance.', 'Yes, because the retailer assented to the terms by remaining silent until after August 1.', 'No, because the offer expressly limited the acceptance to the terms of the offer.', 'No, because the order was withdrawn before the machines were shipped.']\",\"question\":\"In June a computer retailer ordered 100 computers, with various configurations, from a manufacturer, to be delivered by August 1. The order form expressly limited acceptance to the terms of the offer. In July the manufacturer sent a confirmation agreeing to the terms and specifications, but stating that \\\"shipment will be made in two deliveries, half by August 10 and the other half by August 20.\\\" When shipment was not made by August 1, the retailer withdrew the order. Will the manufacturer prevail in enforcing a contract?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}