{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df9b88b8-3ec9-56be-ae8c-ecc7f2da457d","task_key":"all--test--df9b88b8-3ec9-56be-ae8c-ecc7f2da457d","task_revision_id":"1","upstream_id":"","short_description":"Police were conducting a house-to-house search for an escaped convict. During…","config":"all","split":"test","body":"{\"choices\":\"['granted, because the teenager did not have the authority to consent to a search of the house.', 'granted, because the teenager did not have an ownership or shared occupancy interest in the premises.', 'denied, because the police officer reasonably believed that he had permission to enter the dwelling.', 'denied, because the police officer reasonably believed that he had permission to enter the dwelling and reasonably believed that the teenager was competent to give consent.']\",\"question\":\"Police were conducting a house-to-house search for an escaped convict. During the search, a police officer went to a home and rang the doorbell. When a teenage male opened the door, the police officer told him that a convict had escaped and asked if he had seen anyone suspicious in the neighborhood. The teenager said he hadn't but then asked the police officer to come inside to ensure the house was safe. The police officer entered the home and proceeded to look around. When he went into the kitchen, the police officer noticed a bag containing white powder on the counter. Suspecting that the bag contained narcotics, the police officer questioned the teenager as to its contents. The teenager said he didn't live in the house and had never seen it before. The police officer confiscated the bag and took the teenager to the station house for questioning, at which time it was determined that the teenager had escaped from a psychiatric hospital and had broken into the home earlier in the day. Furthermore, the results of a police lab test found that the bag contained heroin. As a result, the police issued an arrest warrant for the owner of the home. He was apprehended and charged with possession of heroin. The homeowner's attorney has filed a motion to suppress, claiming that the heroin confiscated by the police was unlawfully obtained. The motion should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}