{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62b09cc6-77bd-5914-b720-772da87d9ef9","task_key":"default--test--1974","task_revision_id":"1","upstream_id":"1974","short_description":"An owner of an adult bookstore was angry that a city ordinance had been passed…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The owner, because the ordinance infringes on his business rights.', 'The owner, as it is a legitimate business.', 'The city, as they have a right to maintain public decency.', \\\"The city, because the ordinance is designed to protect the city's image.\\\", 'The city, because they have a right to regulate offensive activities.', 'The city, because it does not unreasonably limit alternative avenues of communication.', 'The owner, because the city has not provided sufficient evidence of harm.', 'The city, because zoning laws are within their jurisdiction.', 'The owner, as his First Amendment rights were indeed breached.', \\\"The owner, as his store is not technically an 'adult' establishment.\\\"]\",\"question\":\"An owner of an adult bookstore was angry that a city ordinance had been passed requiring that all \\\"adult\\\" establishments be located in areas zoned industrial. This owner was located in a tourist area downtown and was very profitable. Claiming his First Amendment rights were being violated, he filed suit in court. The court will likely find for:\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}