{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42c5ab6e-aac0-5f59-b86a-c576741af823","task_key":"all--test--42c5ab6e-aac0-5f59-b86a-c576741af823","task_revision_id":"1","upstream_id":"","short_description":"A man offered to sell his barbecue to his neighbor for $100. After receiving the…","config":"all","split":"test","body":"{\"choices\":\"['the man, because the offer to the neighbor terminated when the neighbor learned of the sale to the brother.', 'the man, because there was no consideration to keep the offer open for an extended period of time.', 'the neighbor, because the offer became irrevocable for a reasonable time when the man allowed the neighbor to \\\"think it over. \\\"', 'the neighbor, because, he is a merchant.']\",\"question\":\"A man offered to sell his barbecue to his neighbor for $100. After receiving the man's offer, the neighbor responded, \\\"Let me think it over. \\\" The man then said, \\\"If you say so. \\\" The next day, the man sold the barbecue to his brother for $100. Thereafter, the neighbor decided to accept the man's offer, but learned from a reliable source that the barbecue had been sold to the brother. If the neighbor sues the man for breach of contract, judgment for\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}