{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58b81578-fcb1-5f05-8868-44dc2ba97d3a","task_key":"default--test--1183","task_revision_id":"1","upstream_id":"1183","short_description":"An officer stopped a car for having a burned out headlight and license plate…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The consent was voluntarily given and was not the result of coercion or duress.', 'The consent was involuntary because the officer did not have a warrant to search the car.', 'The consent was involuntary because it was the product of implied duress.', 'The consent was involuntary because the driver did not explicitly say the words \\\"I consent.\\\"', 'The consent was involuntary because the officer found stolen bank checks in the car.', 'The consent was involuntary because the officer did not advise the driver of his right to refuse.', \\\"The consent became voluntary as a matter of law when the driver failed to produce a driver's license.\\\", 'The consent was voluntary because the officer asked politely.', 'The consent was voluntarily given because the driver smiled and was cooperative.', 'The consent was involuntary as the driver was not informed about the consequences of refusal.']\",\"question\":\"An officer stopped a car for having a burned out headlight and license plate light. When the driver could not produce a driver's license, the officer asked him if he minded if they searched the vehicle. The officer did not advise the driver that he had a right to refuse consent or that if he refused his decision would be honored, but there was nothing duplicitous or coercive in the officer's conduct. The driver smiled and said \\\"of course, no problem.\\\" The officer found stolen bank checks under the rear seat. In a later suppression motion, the driver claimed that his consent was not voluntary. What will the court most likely decide on that issue?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}