{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9b30f1a-462a-5dc5-b243-90f9cfb3994c","task_key":"default--test--1092","task_revision_id":"1","upstream_id":"1092","short_description":"During a federal grand jury investigation into money laundering, a man's…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['permit the man to exercise his Fifth Amendment privilege, since such records may have the same incriminating effect as testimony.', 'allow the man to avoid producing the records if he pleads guilty', 'require the man to submit a written statement instead of producing the records', 'suppress the records because they were obtained without a search warrant', 'dismiss the case because the man invoked his Fifth Amendment rights', 'admit the records under the business records exception to the hearsay rule.', 'compel production of the records, because the Fifth Amendment is inapplicable.', 'order a preliminary hearing to determine if the man has grounds to invoke the Fifth Amendment.', 'reject the records because they are irrelevant to the case', 'order a recess to allow the man time to consult with his attorney about the Fifth Amendment']\",\"question\":\"During a federal grand jury investigation into money laundering, a man's business and tax records were subpoenaed by the government. The man invokes his Fifth Amendment privilege against self- incrimination to prevent production of the records in his possession. The court would most likely\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}