{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62debf00-68d3-5fb9-9a62-bf19a94dcf68","task_key":"default--test--1650","task_revision_id":"1","upstream_id":"1650","short_description":"A husband is charged with murder in the shooting death of his brother-in-law. In…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['neither the burden of production nor the burden of persuasion.', 'the burden of proof beyond a reasonable doubt.', 'the burden of production.', 'the responsibility to demonstrate a history of mental illness.', 'the requirement to provide substantial evidence of his insanity at the time of the crime.', 'the obligation to provide expert medical testimony.', 'the duty to prove his innocence.', 'the necessity to convince the jury of his mental instability.', 'the burden of persuasion only.']\",\"question\":\"A husband is charged with murder in the shooting death of his brother-in-law. In the jurisdiction in which the husband is on trial for murder, the defense of insanity is an affirmative defense. The husband pleads insanity as his defense. At trial, in order to prove his defense of insanity, the husband has\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}