{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b3cd66b-d7cb-53b7-9823-a4d2f18a2195","task_key":"default--test--1520","task_revision_id":"1","upstream_id":"1520","short_description":"A defendant, a nurse at a nursing home, is charged with murdering a resident at…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Yes, because the lethal nature of the substance is relevant to the defendant's state of mind and intent.\\\", 'Yes, because the treatise is a public document and therefore exempt from the hearsay rule.', 'No, because the treatise itself must be introduced as an exhibit.', 'Yes, because the prosecutor is allowed to introduce evidence without calling witnesses.', 'No, because the treatise excerpts are considered hearsay and do not qualify for an exception.', 'Yes, although hearsay, under the learned treatise exception to the hearsay rule.', 'Yes, because the prosecution is merely providing evidence that supports their claim.', 'No, because the treatise excerpts were not offered during the examination of a qualified expert.', \\\"No, because the substance's lethal nature is not relevant to the case.\\\"]\",\"question\":\"A defendant, a nurse at a nursing home, is charged with murdering a resident at the home by adding an allegedly lethal substance to the resident's food. At trial, to prove that the substance added to the resident's food could result in death, the prosecutor, without first calling any witnesses, offers to read into evidence several pages from a standard medical treatise that support the prosecution's claim that the substance the defendant added to the food is lethal. Is the evidence offered admissible?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}