{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77a0e074-d321-5d2f-af6f-2c89ace3b4a8","task_key":"default--test--1822","task_revision_id":"1","upstream_id":"1822","short_description":"Generally, erotic materials are protected by the First Amendment for a person's…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Whether the work, taken as a whole, is considered to be offensive or inappropriate by the majority of the population.', 'Whether the work, taken as a whole and judged by contemporary community standards, would be seen by a sensitive or vulnerable person to appeal to the prurient interest.', 'Whether the work, taken as a whole, has been classified as obscene by a government authority.', 'Whether the work, when examined from an individual perspective, incites violent or harmful actions.', 'Whether the work has been properly judged from a national standard to offend sensitive standards of community morals.', 'Whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.', 'Whether the work, in its entirety, promotes or glorifies illegal activities.', 'Whether the work, when viewed in its entirety, defames or slanders any person or group.', 'Whether the work, judged from a global perspective, offends common decency or morality.']\",\"question\":\"Generally, erotic materials are protected by the First Amendment for a person's personal use and possession. However, certain erotic material can reach a point where it steps over the boundaries of First Amendment protection. The Supreme Court has held that if material is obscene it does not mandate First Amendment protection. Which of the following most accurately represents one of the prongs of the three-pronged test enunciated by the U.S. Supreme Court's for determining whether a film, book, video or pictorial is obscene and therefore not subject to protection?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}