{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a7d096e-8d4d-5fd2-9d87-1ccdccf6a8ee","task_key":"default--test--1927","task_revision_id":"1","upstream_id":"1927","short_description":"A legislative assistant to a state senator approached various wealthy lobbyists…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['prevail, because the newspaper disclosed private facts about the plaintiff.', \\\"not prevail, as the senator's public actions and campaign finances are matters of public concern.\\\", 'not prevail, unless he can prove the newspaper intentionally published false information.', \\\"not prevail, unless he is able to prove malice on the defendant's part.\\\", 'prevail, if he can prove the assistant acted with malicious intent.', 'not prevail, because the senator himself unknowingly signed the letters.', 'prevail, because the information was obtained illegally by the assistant.', 'prevail, because the senator had no knowledge of the solicitation of illegal campaign contributions.', 'not prevail, because the newspaper was acting in the public interest by printing the news story.', 'prevail, because the newspaper story placed him in a \\\"false light in the public eye. \\\"']\",\"question\":\"A legislative assistant to a state senator approached various wealthy lobbyists without the senator's knowledge to solicit illegal campaign contributions for the senator's upcoming re-election campaign. The assistant dictated several letters requesting such contributions, which the senator signed without reading, along with a pile of other correspondence. Before the letters were mailed, however, the senator discovered what had happened and then immediately terminated the assistant's employment. Later that same evening, after being notified that he was fired, the assistant returned to the senator's office and used his keys, which had not yet been returned, to enter. The assistant made copies of the letters in question. The following day, the assistant turned over the copies of the letters to an investigative reporter from the local newspaper in the area. The reporter had heard about the assistant's dismissal from another staff member and was curious about all the underlying circumstances. After the assistant provided the reporter with all of the pertinent facts, the reporter wrote a news story regarding the senator's solicitation of illegal campaign contributions. Although the reporter's story was printed in the newspaper, he did not reveal the source of his information. As soon as the publication was made, the FBI initiated an investigation of the senator's campaign finances. If the senator asserts a claim based on invasion of privacy against the newspaper for the publication of the article concerning the solicitation of illegal campaign contributions, the most likely result is that the senator will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}