{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d7f38ce-c3de-51bc-9be7-5dad8dd66509","task_key":"default--test--966","task_revision_id":"1","upstream_id":"966","short_description":"A defendant was charged with and tried for murder. At trial, the defendant…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the defendant has to prove self-defense beyond a reasonable doubt.', 'Yes, because the instruction violates the presumption of innocence in criminal cases.', 'No, because due process does not regulate the burdens of proof in criminal cases.', 'No, because due process does not preclude requiring a defendant to prove self-defense by a preponderance of the evidence.', 'Yes, because due process precludes placing any evidentiary burden on a criminal defendant.', 'Yes, because the prosecution should bear the burden of proving the absence of self-defense.', 'No, because the court has discretion to determine the burdens of proof.', 'Yes, because due process precludes requiring a criminal defendant to bear the burden on an issue that would make an act lawful.', \\\"Yes, because the defendant's right to a fair trial was compromised by the instruction.\\\", 'No, because self-defense is an affirmative defense that the defendant must prove.']\",\"question\":\"A defendant was charged with and tried for murder. At trial, the defendant testified that he had acted in self-defense. The trial court instructed the jury that the defendant had the burden of proving by a preponderance of the evidence that he had acted in self-defense. The defendant objected, arguing that this instruction violated the constitutional requirement that the prosecution prove the offense beyond a reasonable doubt. Did the trial court's instruction violate the defendant's federal constitutional rights?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}