{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cbc4afaa-0262-50f9-accd-bd72d6b18ea5","task_key":"default--test--1177","task_revision_id":"1","upstream_id":"1177","short_description":"During a three-month period, a city was stunned by a series of mysterious…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['uphold the injunction, because the inference of guilt in any prejudicial coverage would deny the defendant his constitutional right of a fair trial.', 'uphold the injunction, because the newspaper can run stories about the case after the trial is over.', \\\"dissolve the injunction, because the newspaper's reporting is protected under the First Amendment.\\\", 'dissolve the injunction, because the freedom of the press should be upheld regardless of the potential impact on the trial.', 'dissolve the injunction, because the newspaper has the right to publish any information it deems newsworthy.', \\\"uphold the injunction, because it protects the defendant's right to a fair and unbiased trial.\\\", 'dissolve the injunction, because a news story about a matter of public interest is absolutely privileged.', 'uphold the injunction, because reference to the defendant by the name pesticide poisoner would be inflammatory and prejudicial.', \\\"uphold the injunction, because it prevents the newspaper from potentially influencing the jury's decision.\\\"]\",\"question\":\"During a three-month period, a city was stunned by a series of mysterious deaths, which claimed the lives of 20 people. Although all the victims had apparently died from poisoning, the police were in a quandary as to who was responsible for the killings. There was finally a breakthrough in the investigation when a police toxicologist determined that all the victims had died after eating poisoned apples which had been purchased at a local market. The apples had all been contaminated with a pesticide. The police then received a letter from a person who claimed responsibility for the poisonings. The letter stated that the killings were in retaliation for the city's new policy of prosecuting toxic polluters. Acting upon an anonymous tip, the police arrested the owner of a pest control company engaged in the manufacture of pesticides, and charged him with 20 counts of murder. Thereafter, the city's largest newspaper ran a series of articles on the killings and referred to the owner as the pesticide poisoner. After the preliminary hearing, the state trial judge issued an exparte injunction against the newspaper prohibiting it from publishing any news during the trial that might be prejudicial to the owner. The newspaper appealed. In light of the U. S. Supreme Court cases to date, the state appellate court should\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}