{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ae3c8e4-8a2d-5efb-9fad-c236ea4a683f","task_key":"all--test--4ae3c8e4-8a2d-5efb-9fad-c236ea4a683f","task_revision_id":"1","upstream_id":"","short_description":"A man told his neighbor, \"If you will mow my lawn, I will pay you $50. \" The…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Yes, because there was an offer for a unilateral contract that became irrevocable prior to the man's attempted revocation.\\\", 'Yes, under the doctrine of quasi-contract.', \\\"No, because the man's revocation was effective, since the neighbor had not completed performance.\\\", \\\"No, because the neighbor had done nothing more than prepare to accept the offer prior to the man's revocation.\\\"]\",\"question\":\"A man told his neighbor, \\\"If you will mow my lawn, I will pay you $50. \\\" The neighbor then purchased gasoline for the mower so he could mow the lawn. The neighbor wheeled his lawn mower to the man's property and started the engine. At that moment, the man suddenly yelled, \\\"I hereby revoke my offer. \\\"If the neighbor thereafter mows the man's lawn, will he recover the $50?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}