{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7cd5bbb4-2c75-5ed4-b739-840ca44afad2","task_key":"all--test--7cd5bbb4-2c75-5ed4-b739-840ca44afad2","task_revision_id":"1","upstream_id":"","short_description":"Police received reliable information that a homeowner had a stolen x-brand…","config":"all","split":"test","body":"{\"choices\":\"['the y-brand stereo.', 'the z-brand stereo.', 'the y-brand and the z-brand stereos.', 'none of the stereos.']\",\"question\":\"Police received reliable information that a homeowner had a stolen x-brand stereo in his possession. The detectives then submitted an affidavit to a neutral magistrate setting forth sufficient underlying circumstances for the issuance of a search warrant. Making a determination of probable cause, the magistrate issued a warrant for the x-brand stereo at the homeowner's address. The police arrived at the homeowner's dwelling, showed him the warrant, and came inside. In the living room, they noticed a y-brand stereo, which had the serial number removed. Upon further inspection, the police determined that the stereo had been stolen from a local electronics store during a recent burglary. Thereupon, the police placed the homeowner under arrest and instructed him to remain seated in the living room while they searched the rest of the home. One of the officers proceeded to the basement where she found the stolen x-brand stereo. She then decided to search the upstairs and came upon a stolen z-brand stereo in the homeowner's second-floor bedroom. The homeowner was subsequently prosecuted for receiving stolen property. At trial, the homeowner moves to prevent introduction of the stereos into evidence. His motion should be granted with respect to\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}