{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05fa3da1-002c-520a-ab21-6c3322549643","task_key":"default--test--1097","task_revision_id":"1","upstream_id":"1097","short_description":"A highway patrol officer stopped a driver for speeding and found that her…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, it must be suppressed because this was an unreasonable, warrantless search.', 'No, the motion will be denied because the initial traffic stop was legal and everything that flowed therefrom was a reasonable step to search for crime.', 'No, the motion will be denied because the search was permissible as incident to a proper arrest.', 'Yes, the motion will be granted because a search of a vehicle requires a search warrant unless there are exceptional exigent circumstances.', 'Yes, the motion will be granted because the officer did not have probable cause to search the car.', 'No, the motion will be denied because the officer had the right to search the car for his own safety.', \\\"No, the motion will be denied because the officer had probable cause to search the car due to the driver's suspended license.\\\", 'Yes, the motion will be granted because the officer did not see the cocaine in plain view before searching the car.', \\\"Yes, the motion will be granted because the officer did not have the driver's consent to search the car.\\\"]\",\"question\":\"A highway patrol officer stopped a driver for speeding and found that her license was suspended. He arrested her for driving while under suspension and secured her in the rear seat of the squad car. He then searched the car where he found several bags of cocaine inside a coat that was setting on the rear seat; he arrested her for possession and possession with intent to deliver cocaine. Prior to trial, the driver moved for suppression of the cocaine. Is it likely that the trial court will grant the suppression motion?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}