{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ca798270-acda-563f-ab03-8d2b7cdf8653","task_key":"all--test--ca798270-acda-563f-ab03-8d2b7cdf8653","task_revision_id":"1","upstream_id":"","short_description":"Law enforcement authorities had probable cause to suspect a man of murder.…","config":"all","split":"test","body":"{\"choices\":\"['No, the entry was authorized by exigent circumstances and the gun was in plain view.', 'No, the entry did not require an arrest warrant because the authorities already had established probable cause.', 'Yes, the entry was a violation of the knock and announce rule.', \\\"Yes, the police must obtain an arrest warrant to enter someone's home for an arrest.\\\"]\",\"question\":\"Law enforcement authorities had probable cause to suspect a man of murder. Without an arrest warrant, they went to his home to arrest him. They knocked and announced, and hearing no response, forcibly broke open the door and entered the house. They didn't find anyone but seized a handgun that was in plain view. The suspect was later arrested for murder; he filed for suppression of the gun. Will the suppression court throw out the gun?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}