{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40c53b2d-efd5-5c30-b7fa-cdc11857501b","task_key":"all--test--40c53b2d-efd5-5c30-b7fa-cdc11857501b","task_revision_id":"1","upstream_id":"","short_description":"An investor sued a corporation for stock fraud. In presenting his case-in-chief,…","config":"all","split":"test","body":"{\"choices\":\"['admitted, under the business records exception.', 'admitted, under the market reports exception.', 'excluded, because the newspaper copy does not fit within any established exception to the hearsay rule.', 'excluded, because it violates the best evidence rule.']\",\"question\":\"An investor sued a corporation for stock fraud. In presenting his case-in-chief, the investor sought to introduce an issue of a newspaper to show the corporation's stock price on that given day. Upon objection by the corporation's attorney, this evidence should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}