{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aafd85c9-129f-54f7-9868-e07f681f4b29","task_key":"default--test--1788","task_revision_id":"1","upstream_id":"1788","short_description":"An accounting professor has been called as a defense expert witness in a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['sustain the objection, because the professor did not rely on the treatise in making his conclusions.', 'sustain the objection, because the statements in the treatise are hearsay not within any recognized exception.', 'overrule the objection and permit the prosecutor to ask the professor about the treatise.', \\\"sustain the objection, because the prosecutor's question is leading.\\\", \\\"sustain the objection, because the treatise is not relevant to the professor's testimony.\\\", 'overrule the objection, because the treatise is considered a reliable source in the field of accounting.', \\\"overrule the objection, because the professor confirmed the treatise's authority.\\\", 'overrule the objection and permit the entire treatise to be introduced into evidence.', \\\"sustain the objection, because the treatise's statements are opinion-based.\\\", 'overrule the objection and permit only the contradictory statements to be read into evidence.']\",\"question\":\"An accounting professor has been called as a defense expert witness in a white-collar criminal case. He testified that accounting records conclusively demonstrated that the defendant had not committed any type of fraud. On cross-examination of the professor, the prosecuting attorney asked him if he was familiar with a particular treatise. The professor responded affirmatively. The attorney then asked him if the treatise was accepted as authoritative and if he was aware of the conclusions regarding the ability of accountants to detect fraud. The professor responded in the affirmative. The prosecutor now attempts to read the following section of the treatise into evidence: \\\"It is virtually impossible to state with conclusive certainty the existence, or lack thereof, of fraud in the majority of white-collar crime cases, given the sophistication of defendants and the variety of accounting methods available. \\\"On objection by the defendant's attorney, the trial judge will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}