{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b81fa0a7-98c0-5497-9d6b-c74561fe95ed","task_key":"default--test--1713","task_revision_id":"1","upstream_id":"1713","short_description":"Cooper is suing March for money damages. Because he believes portions of March's…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Cooper may use the deposition at trial, but only if he can prove that March's statements are unreliable.\\\", \\\"Cooper may use the deposition at trial, but only if March's attorney is present during the reading.\\\", 'Cooper may not use the deposition at trial, as it is considered hearsay.', 'Cooper may not use the deposition at trial, as this would make March his witness and immune to impeachment.', 'Cooper may use the deposition at trial, but must read the entire deposition, not just selected parts.', \\\"Cooper may not use the deposition at trial, as it would violate March's right to confrontation.\\\", \\\"Cooper may use the deposition at trial, but only to contradict or impeach March's prior inconsistent statements or pleadings.\\\", 'Cooper may use the deposition at trial, but, if requested, he must read all parts that in fairness ought to be considered with the part introduced.', 'Cooper may not use the deposition at trial, unless March is unavailable to testify.', 'Cooper may not use the deposition at trial, as March is able to testify and no exceptional circumstances exist.']\",\"question\":\"Cooper is suing March for money damages. Because he believes portions of March's deposition are highly favorable to his case, Cooper's attorney intends to read parts of the deposition at trial instead of calling March to the stand. March objects to Cooper's use of the deposition at trial. What is the court's likely ruling?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}