{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bf6f961d-3520-5052-8adf-a65714089312","task_key":"all--test--bf6f961d-3520-5052-8adf-a65714089312","task_revision_id":"1","upstream_id":"","short_description":"During a federal grand jury investigation into money laundering, a man's…","config":"all","split":"test","body":"{\"choices\":\"['compel production of the records, because the Fifth Amendment is inapplicable.', 'order a preliminary hearing to determine if the man has grounds to invoke the Fifth Amendment.', 'permit the man to exercise his Fifth Amendment privilege, since such records may have the same incriminating effect as testimony.', 'admit the records under the business records exception to the hearsay rule.']\",\"question\":\"During a federal grand jury investigation into money laundering, a man's business and tax records were subpoenaed by the government. The man invokes his Fifth Amendment privilege against self- incrimination to prevent production of the records in his possession. The court would most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}