{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfb3f893-64a6-5f77-b008-1f2582d820da","task_key":"all--test--dfb3f893-64a6-5f77-b008-1f2582d820da","task_revision_id":"1","upstream_id":"","short_description":"An on-duty Police Officer hears breaking glass and a blaring alarm, and sees…","config":"all","split":"test","body":"{\"choices\":\"['Denied, because the Officer had probable cause to search the trunk.', 'Denied, because a search incident to a valid arrest in an automobile always includes the trunk of the automobile.', \\\"Granted, because the trunk area was out of the range of Stan's control.\\\", 'Granted, because the arrest was illegal and therefore the evidence is considered to be fruit of the poisonous tree.']\",\"question\":\"An on-duty Police Officer hears breaking glass and a blaring alarm, and sees Stan run out of a darkened liquor store at 3 a.m. Stan runs around the corner, out of Officer's view, and hops into a car. Officer sees Stan drive away and pursues, lights flashing. Stan pulls over and Officer arrests him. Officer sees nothing in the passenger compartment, but pops the trunk and finds cash and a baseball bat. Stan's motion to suppress the cash and baseball bat as evidence will likely be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}