{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c292224c-ddfb-51f8-b48b-19093f010a46","task_key":"all--test--c292224c-ddfb-51f8-b48b-19093f010a46","task_revision_id":"1","upstream_id":"","short_description":"A police officer stopped a driver who had run a red light. Upon approaching the…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The statement should be allowed, because although the driver was in custody, the officer's spontaneous utterance upon smelling alcohol did not rise to the level of interrogation.\\\", 'The statement should be allowed, because the driver was not in custody for Miranda purposes when the admission was made.', \\\"The statement should be suppressed both in the prosecution's case-in-chief and as impeachment evidence, even if the driver testifies.\\\", \\\"The statement should be suppressed in the prosecution's case-in-chief, but it may be prosecution's case-in-chief, but it may be used as impeachment evidence if the driver testifies.\\\"]\",\"question\":\"A police officer stopped a driver who had run a red light. Upon approaching the car, the officer noticed a strong odor of alcohol and immediately asked whether the driver had been drinking. The driver admitted having had several alcoholic drinks that evening. The driver, charged with driving while intoxicated, moved to suppress the officer's testimony regarding the driver's statement about his drinking. The driver argued that the officer had elicited the statement without providing the requisite Miranda warnings. The prosecutor has responded that the statement The prosecutor has responded that the statement should be allowed in the prosecution's case-in- chief or, at a minimum, should be allowed as impeachment in the event the driver testifies and denies drinking. How should the court rule regarding the driver's statement admitting his drinking?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}