{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"53f7816b-7d3c-5db2-aff0-0a33998f7176","task_key":"default--test--1419","task_revision_id":"1","upstream_id":"1419","short_description":"A college student initiated a criminal case against his former roommate,…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Yes, because the student's reputation for dishonesty directly relates to the tort of malicious prosecution.\\\", 'No, because reputation evidence is considered too prejudicial to be admissible.', 'Yes, because dishonesty at any time in the past is always relevant to a malicious prosecution complaint.', \\\"Yes, because the student's dishonesty directly affects the credibility of his initial criminal complaint.\\\", 'Yes, because reputation evidence, just like most character evidence, is virtually always admissible.', 'Yes, because even though character evidence is too collateral to be admitted for circumstantial evidence, it is admissible if it is directly at issue in the case.', 'No, because character evidence is only admissible if it is directly related to the act in question.', 'No, because the past reputation of a person is not considered as evidence in court.', 'Yes, because while most character evidence is not admissible, some is, and reputation evidence is always admissible.', 'No, because character evidence is never admissible in any case.']\",\"question\":\"A college student initiated a criminal case against his former roommate, claiming the former roommate snuck in and removed his state-of-the-art home theater equipment from his house. The college student took the complaint papers that were approved by the prosecution and filed them with a magistrate court. An ongoing police investigation later determined, however, that the college student was a vengeful person not averse to creating stories, and that he and the former roommate had been arguing over several related issues. They also determined that he never had any quality sound or video equipment. The police dropped the prosecution, and the criminal case was dismissed at the preliminary hearing. When the former roommate filed a civil case against the college student alleging the tort of malicious prosecution, will evidence of the student's reputation for dishonesty be admissible and why?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}