{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa404a9d-02c2-5176-811d-d98a5f36f1fa","task_key":"default--test--1226","task_revision_id":"1","upstream_id":"1226","short_description":"A man outraged by the recent church decision to clear a famous philosopher of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The statute is not a violation of the First Amendment because it does not directly regulate the content of speech.', 'The statute, as applied to the man, is a violation of his First Amendment rights because it restricts his ability to express his thoughts and ideas.', 'The statute, as applied to the man, does not violate his First Amendment rights because his proposed speech is not political and, therefore, not among the classes of speech that are protected.', 'The statute is a prior restraint on speech, which unconstitutionally vests unfettered discretion in the school board to decide who may use university lecture halls.', \\\"The statute is a violation of the man's freedom of speech because it grants the school board the power to censor speech.\\\", 'The statute is unconstitutional because it gives the school board too much power to determine what constitutes a \\\"worthwhile program of public benefit.\\\"', 'Indirect speech, regulations are only permissible if necessary to serve compelling state interests.', 'The statute is unconstitutionally overbroad, because it may result in the exclusion of protected speech as well as unprotected speech.', 'The statute, as applied to the man, is not a violation of his First Amendment rights because his proposed speech can be considered offensive or harmful to the public.', 'The statute is unconstitutional because it may result in the suppression of speech that criticizes or challenges established beliefs or institutions.']\",\"question\":\"A man outraged by the recent church decision to clear a famous philosopher of charges of heresy, decided to present a lecture, open to the public, disproving the philosopher's theories. A state statute provides that: \\\"state universities can permit the use of their lecture halls to the public for worthwhile programs of public benefit, upon approval of the school board. \\\"The appropriate school board refused to make a university lecture hall available to the man on the grounds that the proposed lecture was not of worthwhile benefit to the public. As a result, the man brought suit in a state court against the school board and requested injunctive relief requiring the board to allow him the use of the lecture hall. The trial court denied relief and dismissed the suit. The judgment was affirmed by the state appellate court, and is now before the U. S. Supreme Court. In analyzing the state statute, which of the following statements is least accurate?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}