{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6ff65703-3f6d-5f77-ae6a-1c97d7b7a116","task_key":"all--test--6ff65703-3f6d-5f77-ae6a-1c97d7b7a116","task_revision_id":"1","upstream_id":"","short_description":"A man was in an accident while driving his car on July 3. The other driver sent…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the renewal premium was placed in the mailbox before the date of expiration, and the notice did not say that the check had to be received physically before there could be a renewal.', 'Yes, because a policy cannot be lapsed for nonpayment without a final 30 day notice of impending lapse being sent.', \\\"No, the man knew that the policy would not be renewed if did not get the payment into the insurer's possession by the due date of June 28.\\\", 'No, because the mail box rule is not applicable in a situation where an insurance policy must be renewed.']\",\"question\":\"A man was in an accident while driving his car on July 3. The other driver sent him a notice of injuries and demanded damages. The man notified his insurance carrier but was told that his policy had lapsed on July 2. He had earlier received a renewal notice with a bill listing a due date of June 28 and an expiration date of the policy on July 2. The notice welcomed payment by mail, and did not say that renewal was conditioned on actual physical receipt of the check. He had put the check and form in the U.S. mail on June 28, but the insurer did not receive it until July 5. The insurer reinstated the policy as of July 5, leaving a coverage gap from July 2 through July 5, according to the company. Will the courts likely hold that the policy was still effective on July 3, the date of the accident?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}