{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dcdeb8c-cb31-5907-9d6b-3d297f6adbef","task_key":"all--test--4dcdeb8c-cb31-5907-9d6b-3d297f6adbef","task_revision_id":"1","upstream_id":"","short_description":"A defendant parked his car illegally in a handicapped parking space while he…","config":"all","split":"test","body":"{\"choices\":\"['the heroin only.', 'the marijuana only.', 'both the heroin and the marijuana.', 'neither the heroin nor the marijuana.']\",\"question\":\"A defendant parked his car illegally in a handicapped parking space while he went into a store to get a cup of coffee. The defendant was inside the store waiting for his coffee when a member of the city's parking enforcement department drove by the vehicle. Seeing that there was no handicapped sticker on the car, the parking official stopped and started ticketing the vehicle. When the defendant came outside, he got into an argument with the parking official. After the defendant became verbally abusive, the parking official told him to desist or he was going to impound the vehicle. After the defendant continued his badgering, the parking official told him that he was impounding the vehicle and called the station house for officer assistance. A few minutes later, a police officer arrived on the scene. He asked the defendant for identification and then proceeded to frisk him for weapons. The officer found a clear vial containing heroin in the defendant's back pocket. He then placed the defendant under arrest. Thereupon, the parking official went into the defendant's car and began an inventory search. In this jurisdiction, parking enforcement personnel have statutory authority to conduct inventory searches of impounded vehicles. The parking official did not find anything of interest. After the police officer placed the defendant in the patrol car, the officer asked the parking official if he could search the vehicle, to which the parking official responded yes. The officer looked under the floor mat and found a bag containing marijuana. The defendant was taken to the station house and charged with unlawful possession of heroin and marijuana. Prior to trial, the defense filed a motion to exclude the heroin and marijuana from being admitted into evidence. The motion will be granted with respect to\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}