{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed2397ce-f66c-5e7b-a29a-c90ff6920659","task_key":"default--test--1641","task_revision_id":"1","upstream_id":"1641","short_description":"A state law imposes penalties for \"any public statement containing false or…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The airline had no intent to deceive.', 'The state law is vague and unclear.', 'Its statement about the safety record was made without malice.', 'The state law violates the principle of freedom of speech.', 'The state law is overbroad.', 'The state law is a prior restraint.', \\\"The airline's statement was based on factual data from a trusted source.\\\", \\\"The airline's statement is a form of protected commercial speech.\\\", 'Its statement about the safety record was protected noncommercial speech.', 'The airline was exercising its right to competitive advertising.']\",\"question\":\"A state law imposes penalties for \\\"any public statement containing false or misleading information about a service or product.\\\" An airline falsely claimed in an advertisement that its competitor had an inferior safety record. The claim was based on erroneous information, found on the website of a nonprofit consumer advocacy group, that the airline assumed to be true. The airline was charged under the state law for making a false statement. No federal statute applies. Which of the following best supports the airline in a defense based on the First Amendment?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}