{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"241b17a1-c4b2-52c8-9e81-cd1e43f20dc0","task_key":"all--test--241b17a1-c4b2-52c8-9e81-cd1e43f20dc0","task_revision_id":"1","upstream_id":"","short_description":"An officer stopped a car for having a burned out headlight and license plate…","config":"all","split":"test","body":"{\"choices\":\"['The consent was voluntarily given and was not the result of coercion or duress.', 'The consent was involuntary because it was the product of implied duress.', 'The consent was involuntary because the officer did not advise the driver of his right to refuse.', \\\"The consent became voluntary as a matter of law when the driver failed to produce a driver's license.\\\"]\",\"question\":\"An officer stopped a car for having a burned out headlight and license plate light. When the driver could not produce a driver's license, the officer asked him if he minded if they searched the vehicle. The officer did not advise the driver that he had a right to refuse consent or that if he refused his decision would be honored, but there was nothing duplicitous or coercive in the officer's conduct. The driver smiled and said \\\"of course, no problem.\\\" The officer found stolen bank checks under the rear seat. In a later suppression motion, the driver claimed that his consent was not voluntary. What will the court most likely decide on that issue?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}