{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da6fe5bc-1585-5358-be4f-445065576daf","task_key":"default--test--1233","task_revision_id":"1","upstream_id":"1233","short_description":"An on-duty Police Officer hears breaking glass and a blaring alarm, and sees…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Denied, because the Officer had a reasonable suspicion that Stan was involved in a crime.', 'Granted, because the arrest was illegal and therefore the evidence is considered to be fruit of the poisonous tree.', 'Granted, because the Officer did not have a search warrant to check the trunk of the car.', 'Granted, because the Officer did not have a valid reason to pull Stan over in the first place.', 'Denied, because a search incident to a valid arrest in an automobile always includes the trunk of the automobile.', 'Denied, because the Officer had probable cause to search the trunk.', 'Denied, because the Officer had the right to search the car without a warrant.', \\\"Granted, because the trunk area was out of the range of Stan's control.\\\", 'Denied, because the Officer was in the pursuit of his duties and the search was incidental to the arrest.', 'Granted, because the Officer did not observe Stan commit any crime.']\",\"question\":\"An on-duty Police Officer hears breaking glass and a blaring alarm, and sees Stan run out of a darkened liquor store at 3 a.m. Stan runs around the corner, out of Officer's view, and hops into a car. Officer sees Stan drive away and pursues, lights flashing. Stan pulls over and Officer arrests him. Officer sees nothing in the passenger compartment, but pops the trunk and finds cash and a baseball bat. Stan's motion to suppress the cash and baseball bat as evidence will likely be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}