{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e803334-d643-54f5-96e4-6ace30edbeef","task_key":"default--test--1938","task_revision_id":"1","upstream_id":"1938","short_description":"A nightclub owner applied for a required zoning permit to open a nude-dancing…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the city council has the right to consider the views of its citizens.', \\\"No, because the city council's consideration of the organization's views is biased and unfair.\\\", \\\"Yes, because the organization's lobbying efforts are a lawful expression of their views.\\\", \\\"No, because the organization's lobbying efforts infringe on the owner's right to conduct business.\\\", \\\"No, because the organization does not seek a reasonable time, place, and manner regulation of nude dancing, but instead seeks a total ban on the owner's opening any type of nude-dancing facility at any time or in any place in the city.\\\", 'Yes, because the action is not ripe.', 'No, because the owner has a constitutional right to free expression, which includes operating a nude-dancing facility.', 'Yes, because the First and Fourteenth Amendments do not protect obscenity, and nude dancing is obscene.', \\\"Yes, because the owner's lawsuit interferes with the city council's decision-making process.\\\", 'No, because nude dancing is symbolic speech and is therefore protected by the First and Fourteenth Amendments.']\",\"question\":\"A nightclub owner applied for a required zoning permit to open a nude-dancing nightclub in the theater district of a city. An organization of influential city residents began an intensive lobbying effort to persuade the city council to deny the owner a permit to operate any type of nude- dancing facility at any time or in any place in the city. The owner has sued the city in an appropriate federal court, seeking an injunction that would prohibit the city council from considering the organization's views, on the ground that if the organization is successful in its lobbying efforts, the owner's First and Fourteenth Amendment rights would be violated. The city has moved to dismiss the action. Should the court dismiss the owner's action? \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}