{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39dee535-d03b-5613-9159-ff6ca552f4ae","task_key":"default--test--1161","task_revision_id":"1","upstream_id":"1161","short_description":"In a criminal prosecution the government must prove that the defendant affixed a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The testimony is inadmissible because evidence of prior habit will only distract the jury from deciding whether the defendant actually acted that way on the specified dates.', 'The testimony is inadmissible because it is not direct evidence that the defendant used the stamp on the fraudulent letters.', 'The testimony is inadmissible because the co-worker does not have a direct knowledge of the defendant using the stamp on the fraudulent letters.', 'The testimony is inadmissible because evidence of habit is not admissible in a criminal case under the Federal Rules of Evidence.', 'The testimony is admissible because it proves that the defendant had access to the stamp.', 'The testimony is admissible because it proves a pattern of behavior that the defendant maintained control over the stamp.', 'The testimony is admissible because it proves that the defendant was the only person who could have used the stamp.', \\\"The testimony is inadmissible because the co-worker's observation is considered hearsay under the Federal Rules of Evidence.\\\", 'The testimony is admissible because habit and routine practice are admissible under the Federal Rules of Evidence.', \\\"The testimony is admissible because it provides the jury with information about the defendant's routine, which is relevant to the case.\\\"]\",\"question\":\"In a criminal prosecution the government must prove that the defendant affixed a rubber-stamp containing his signature to certain fraudulent letters mailed to the victim. The defendant denied that he stamped the letters. There were no witnesses to the defendant stamping the letters. The prosecution attempted to present a co-worker to testify that defendant maintained sole control of the stamp and used it once daily at 3 p.m. to sign all outgoing letters, and then immediately returned it to his safe. The co-worker would testify that he saw the defendant using the stamp in that habitual manner over a period of many years. The defense objected because this did not prove that the defendant's practice was followed on the dates in question. Under the Federal Rules of Evidence, what will the court rule?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}