{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6ed45f5d-c014-56e5-b519-39a9c0d7ec45","task_key":"default--test--1269","task_revision_id":"1","upstream_id":"1269","short_description":"A highway patrol officer stopped a driver for speeding and found that the person…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, the motion will be granted because a search of a vehicle requires a search warrant unless there are exceptional exigent circumstances.', \\\"Yes, the motion will be granted because the officer did not have the driver's consent to search the car.\\\", 'No, the motion will be denied because the officer had the right to search the car for officer safety reasons.', \\\"No, the motion will be denied because the officer had a reasonable suspicion to search the car due to the driver's suspended license.\\\", 'No, the motion will be denied because the officer had probable cause to search the car after finding out about the suspended license.', 'No, the motion will be denied because this was a valid search incident to an arrest.', 'Yes, it must be suppressed because she was secured in the squad car and the police could not expect to find evidence relating to the current arrest in the car.', 'Yes, the motion will be granted because the cocaine was found in a personal item, not in plain sight.', \\\"No, the motion will be denied because there is no expectation of privacy toward property inside one's car.\\\", 'Yes, the motion will be granted because the officer did not have a warrant to search the jacket on the back seat.']\",\"question\":\"A highway patrol officer stopped a driver for speeding and found that the person had a suspended license. He allowed her husband to drive the car home but arrested her for driving under suspension. After securing her inside the squad car, he conducted a search of her vehicle. The officer found a bag of cocaine inside a zippered jacket on the back seat. He arrested her for possession and possession with intent to deliver a controlled substance. Prior to trial, the driver moved for suppression of the cocaine. Is it likely that the trial court will grant the suppression motion?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}