{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9fcd0fcb-6c4f-595f-8dcc-473fc9740a12","task_key":"all--test--9fcd0fcb-6c4f-595f-8dcc-473fc9740a12","task_revision_id":"1","upstream_id":"","short_description":"On Thursday, March 7, an owner's home was destroyed by fire. The owner's house…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, because the owner's statement to her is non-hearsay.\\\", 'admissible, because a letter properly mailed is presumed received in due course.', \\\"inadmissible, because the owner's statement to the bookkeeper is not the best evidence.\\\", \\\"inadmissible, because the owner's statement to the bookkeeper is hearsay not within any exception.\\\"]\",\"question\":\"On Thursday, March 7, an owner's home was destroyed by fire. The owner's house was insured against such fire loss by an insurance company. According to the policy contract, the insured was required to notify the insurance company within 10 days following any loss due to fire or other catastrophe. The owner sued the insurance company to recover $750,000, which was the face amount that his home was insured for. The insurance company admitted that the owner's house was insured for that amount, though the company denied liability because it claimed that it had not received notice of the fire within the 10-day period as required by the policy. At trial, the owner calls his bookkeeper to testify. The bookkeeper proposes to testify that on March 11, the owner handed her an envelope and told her it contained a notice form for the insurance company, and that she mailed the envelope on his behalf. If the bookkeeper intends to establish that the envelope that the owner handed her contained the notice of fire loss, her proposed testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}