{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45cfadef-cf6b-5444-96bd-158e2093f25c","task_key":"default--test--1546","task_revision_id":"1","upstream_id":"1546","short_description":"A plaintiff was the varsity basketball coach of the boys' team at a high school.…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"admissible, as it provides a motive for the plaintiff's dismissal.\\\", 'inadmissible to prove character, but admissible for the purpose of showing intent or common scheme.', 'admissible, as substantive evidence.', 'admissible, for impeachment purposes only.', 'inadmissible, as it is not relevant to the case.', \\\"admissible, as it directly contradicts the plaintiff's own testimony.\\\", 'inadmissible, as it is prejudicial to the plaintiff.', 'admissible, as character evidence in a defamation suit.', 'inadmissible, as it constitutes hearsay.']\",\"question\":\"A plaintiff was the varsity basketball coach of the boys' team at a high school. After the second game of the season, the plaintiff was suddenly dismissed as the coach for undisclosed reasons. After the dismissal was made public, the local newspaper ran an article discussing the plaintiff's tenure as coach. The article, which was written by the sports editor, stated that \\\"The plaintiff was a bad role model and morally unfit for the coaching job. \\\"The plaintiff brought suit against the newspaper for defamation. At trial, the plaintiff called a player on the basketball team to testify that \\\"the plaintiff was a good role model and respected by the players. \\\" In presenting its evidence, the newspaper called a witness to testify. He proposes to testify that he saw the plaintiff give beer to players on three occasions after victories. Upon objection by the plaintiff's attorney, witness's testimony is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}