{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b25d40ec-d293-58df-83ef-b6953272cf07","task_key":"default--test--1828","task_revision_id":"1","upstream_id":"1828","short_description":"A man and a woman were partners in an automobile leasing agency. For some time,…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Conspiracy and solicitation.', 'Solicitation and conspiracy.', 'Conspiracy, solicitation, embezzlement, and murder.', 'Embezzlement and solicitation.', 'Murder.', 'No additional crimes, only embezzlement.', 'Solicitation, embezzlement, and murder.', 'Conspiracy, embezzlement, and murder.', 'Conspiracy and murder.']\",\"question\":\"A man and a woman were partners in an automobile leasing agency. For some time, the man had been embezzling money from the agency. The woman became suspicious and told the man that she wanted to engage an accounting firm to audit the agency's books. In a panic to prevent the audit and avoid being discovered, the man decided to have the woman killed. He contacted a professional killer and hired him to kill the woman for $10,000. A short time later, the woman died in a car crash arranged by the killer. In addition to embezzlement, of what crime or crimes may the man properly be convicted?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}