{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbcc8cd7-0040-5e87-822a-c176e3c1b0d9","task_key":"all--test--bbcc8cd7-0040-5e87-822a-c176e3c1b0d9","task_revision_id":"1","upstream_id":"","short_description":"A newspaper printed an article stating that a local doctor was afflicted with a…","config":"all","split":"test","body":"{\"choices\":\"['Yes, the doctor is a professional figure in the community and he must accept the risk of false reporting, especially where he cannot prove any special damages.', 'Yes, the newspaper is allowed to make mistakes as long as there is no malice, and the case must be dismissed as a matter of law.', 'No, this fits the requirements for slander per se and there is a valid claim of malice stated against the paper.', 'No, this fits the requirements for libel per se, which is so inflammatory that the doctor does not have to prove special damages.']\",\"question\":\"A newspaper printed an article stating that a local doctor was afflicted with a venereal disease and had volunteered for a program studying the various cures for the disease. The article was false, and was printed without adequate research. The doctor sued the newspaper company, claiming damages per se. The paper defended on the basis that this was a public interest article, that the paper was not reckless or malicious in its investigation and reporting, and that the doctor could not prove any special damages. Will the court adopt the defense reasoning and dismiss the case for failure to state a claim?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}