{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aacb351d-d736-5f1d-b8ef-ab0b079fd1d7","task_key":"default--test--1246","task_revision_id":"1","upstream_id":"1246","short_description":"The defendant has been charged with murder in the killing of his wife. The…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The defendant acted without deliberation and premeditation.', 'The defendant acted in self-defense.', 'The defendant was provoked by accidentally seeing his wife having sexual relations with his best friend.', \\\"The defendant was provoked by his wife's refusal to prepare his favorite meal.\\\", 'The defendant acted in an overpowering heat of passion.', 'The defendant acted several days after the provoking events.', 'The defendant was provoked by a minor disagreement with his wife.', 'The defendant acted while under the influence of alcohol or drugs.']\",\"question\":\"The defendant has been charged with murder in the killing of his wife. The defense attorney is preparing a defense to show facts sufficient to support a verdict of voluntary manslaughter. Counsel has to prove all of the necessary elements during trial in order to support a voluntary manslaughter charge to the jury. Which of the following would not be proper proof to establish a voluntary manslaughter charge to the jury?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}