{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f24b7b9-77b6-56f5-85eb-82211c4e19af","task_key":"all--test--5f24b7b9-77b6-56f5-85eb-82211c4e19af","task_revision_id":"1","upstream_id":"","short_description":"A motel advertises the showing of pornographic, or adult movies in the privacy…","config":"all","split":"test","body":"{\"choices\":\"['sustained, because a state can use its police power to prohibit the showing of pornography in public areas.', 'sustained, because a state may use local standards in determining whether a movie has redeeming literary, artistic, political, or scientific merit.', 'overturned, because his prosecution violates the right of consenting adultsto view such films in private.', 'overturned, because the First and Fourteenth Amendments prohibit the suppression of sexually oriented materials on the basis of their allegedly obscene contents.']\",\"question\":\"A motel advertises the showing of pornographic, or adult movies in the privacy of each room. The motel has a strict policy permitting adults only to occupy the rooms. The state has recently enacted a statute that prohibits the showing of any obscene film in an area open to the public. The owner of the motel is prosecuted for violating the statute by showing pornographic movies in the motel rooms. On appeal, the owner's conviction will probably be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}