{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f52017d-89b9-57d9-8524-3bbb12b18b5c","task_key":"all--test--3f52017d-89b9-57d9-8524-3bbb12b18b5c","task_revision_id":"1","upstream_id":"","short_description":"A man brought a federal diversity action against his insurance company, alleging…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because a motion for judgment as a matter of law must first be made at the close of the plaintiff's case-in-chief.\\\", \\\"No, because whether the man's behavior was unduly risky is a question of fact for the jury to resolve.\\\", \\\"Yes, because the company's uncontradicted evidence of the man's unduly risky behavior means that no reason-able jury could find that the policy covers his injuries.\\\", \\\"Yes, because the man waived his right to rebut the company's evidence by not addressing the Òunduly riskyÓ policy provision in his case-in-chief.\\\"]\",\"question\":\"A man brought a federal diversity action against his insurance company, alleging that the company had breached its duty under his insurance policy by refusing to pay for his medical expenses resulting from a mountain-biking accident. At the jury trial, the man presented evidence that he had paid all premiums on the insurance policy and that the policy cov-ered personal-injury-related medical expenses arising from accidents. After he rested his case, the company presented evidence that a provision of the policy excluded payment for injury-related expenses resulting from an insured's Òunduly riskyÓ behavior. The company also presented a witness who testified that the accident had occurred in an area where posted signs warned bikers not to enter. The man did not cross-examine the witness. After resting its case, the company moved for judgment as a matter of law. Should the court grant the motion?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}