{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0f9db24-8adf-502d-987e-dbd15285d301","task_key":"default--test--1825","task_revision_id":"1","upstream_id":"1825","short_description":"A plaintiff is the beneficiary of a policy issued by an insurance company…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['inadmissible, due to lack of relevance.', 'admissible as former testimony.', 'admissible, as it could demonstrate a potential motive.', 'inadmissible, because of double jeopardy.', 'admissible, as an exception to the hearsay rule.', 'admissible, as part of the res gestae.', 'inadmissible, because it is prejudicial.', 'admissible as a declaration against interest.', 'inadmissible, because of collateral estoppel.']\",\"question\":\"A plaintiff is the beneficiary of a policy issued by an insurance company insuring the life of his wife, now deceased. The policy contains a clause providing that double indemnity is payable in the event that death of the insured \\\"results directly, and independently of all other causes, from bodily injury effected solely through external violent and accidental means. \\\"The plaintiff's wife met her death in the silage shed of her farm. The death resulted from wounds caused by the discharge of a double-barreled shotgun. The plaintiff was arrested and prosecuted for the murder of his wife. After a lengthy trial, the plaintiff was acquitted of the charge. After the insurance company refused to pay the plaintiff's insurance claim, the plaintiff instituted this civil action. The complaint sets forth the policy, alleges the facts surrounding the death of the insured, avers that the death was within the policy provisions for double indemnity payment, and demands judgment accordingly. At trial, the insurance company's attorney proposes to introduce excerpts from the plaintiff's murder trial. The plaintiff's attorney objects to the introduction of such evidence. The trial judge should rule the proffered evidence\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}