{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe8c3026-8f23-58fd-ad6f-9b02a285954b","task_key":"all--test--fe8c3026-8f23-58fd-ad6f-9b02a285954b","task_revision_id":"1","upstream_id":"","short_description":"A legislative assistant to a state senator approached various wealthy lobbyists…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because the newspaper story placed him in a \\\"false light in the public eye. \\\"', 'prevail, because the newspaper disclosed private facts about the plaintiff.', \\\"not prevail, unless he is able to prove malice on the defendant's part.\\\", 'not prevail, because the newspaper was acting in the public interest by printing the news story.']\",\"question\":\"A legislative assistant to a state senator approached various wealthy lobbyists without the senator's knowledge to solicit illegal campaign contributions for the senator's upcoming re-election campaign. The assistant dictated several letters requesting such contributions, which the senator signed without reading, along with a pile of other correspondence. Before the letters were mailed, however, the senator discovered what had happened and then immediately terminated the assistant's employment. Later that same evening, after being notified that he was fired, the assistant returned to the senator's office and used his keys, which had not yet been returned, to enter. The assistant made copies of the letters in question. The following day, the assistant turned over the copies of the letters to an investigative reporter from the local newspaper in the area. The reporter had heard about the assistant's dismissal from another staff member and was curious about all the underlying circumstances. After the assistant provided the reporter with all of the pertinent facts, the reporter wrote a news story regarding the senator's solicitation of illegal campaign contributions. Although the reporter's story was printed in the newspaper, he did not reveal the source of his information. As soon as the publication was made, the FBI initiated an investigation of the senator's campaign finances. If the senator asserts a claim based on invasion of privacy against the newspaper for the publication of the article concerning the solicitation of illegal campaign contributions, the most likely result is that the senator will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}