{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"324dfc59-2458-59a9-8a6b-4b67363fb3b9","task_key":"all--test--324dfc59-2458-59a9-8a6b-4b67363fb3b9","task_revision_id":"1","upstream_id":"","short_description":"The police suspected a woman of dealing in stolen credit cards. An undercover…","config":"all","split":"test","body":"{\"choices\":\"['No, because it was immediately apparent that the cards were stolen.', 'No, because the woman assumed the risk that the officer would seize items in plain sight.', \\\"Yes, because the woman's rights were violated when the officer lifted the papers.\\\", \\\"Yes, because the woman's rights were violated when the officer obtained entry by deceit.\\\"]\",\"question\":\"The police suspected a woman of dealing in stolen credit cards. An undercover police officer knocked on the woman's door and told her that he was willing to pay for usable credit cards. When the woman asked if he was a police officer, the officer replied, \\\"No,\\\" giving her a false name and saying, \\\"You can call Harvey €”he'll vouch for me.\\\" The woman admitted him to her house. After she left the room to call Harvey, the officer lifted some papers off a desk and underneath discovered three credit cards bearing different names. He seized the cards, and when the woman returned, he arrested her. The woman has moved to suppress the evidence seized on the ground that her Fourth Amendment rights were violated. Should the court grant the motion to suppress?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}