{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab009832-7d5d-593f-b6c9-9bce735e6d97","task_key":"all--test--ab009832-7d5d-593f-b6c9-9bce735e6d97","task_revision_id":"1","upstream_id":"","short_description":"Jan sues the Chicago Times for defamation in a Florida circuit court. At trial,…","config":"all","split":"test","body":"{\"choices\":\"['call as a witness an employee of the Times to testify that the proffered newspaper was in fact published by the Times.', 'have a certification affixed to the newspaper signed by an employee of the Times attesting to its authenticity.', 'establish that the newspaper has been properly certified under the law of the jurisdiction where the newspaper was published.', 'do nothing because the newspaper is self-authenticating.']\",\"question\":\"Jan sues the Chicago Times for defamation in a Florida circuit court. At trial, Jan wishes to offer into evidence a copy of the edition of the Times containing the allegedly libelous article. Before the newspaper may be admitted, Jan must\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}