{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab0ee40e-ce46-5f78-92b3-ad0399be1e4e","task_key":"default--test--1534","task_revision_id":"1","upstream_id":"1534","short_description":"A man wearing a jacket and jeans walked along a city street at night in a high…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the accumulation of factors all added up to reasonable suspicion for a stop and articulable suspicion for a frisk.', \\\"No, because the man's suspicious behavior and the bulge in his pocket gave the police probable cause to search him.\\\", 'Yes, because the police lacked reasonable suspicion to stop the man simply because he crossed the street and walked faster away from them.', \\\"Yes, because the police didn't have a warrant to search the man.\\\", \\\"Yes, because the heavy object in the man's pocket could have been anything and didn't necessarily indicate criminal activity.\\\", 'No, because the police had probable cause to make an arrest when the man began to walk away from them and walk even faster as they followed him.', 'No, because the police had witnessed a crime, jaywalking, which gave them the authority to stop the man.', \\\"Yes, because they didn't know if this was the same man, they observed nothing illegal except jaywalking, and he had a right to avoid them under the circumstances.\\\", 'Yes, because the police had no evidence that the man was the same one the resident had reported.', 'No, because the man matched the description of the suspected drug dealer and was in a high crime area.']\",\"question\":\"A man wearing a jacket and jeans walked along a city street at night in a high crime area known for drug trafficking and gun violence. He saw two police officers walking toward him. The officers were checking on a 911 call from a resident who observed a man in jeans and a jacket apparently selling drugs to people on the street. When the police first saw him, he was walking alone, in the area that the woman identified. As they approached, he suddenly crossed the street by jaywalking. They followed, and approaching from behind, asked him to stop. He started walking faster, away from them. The officers saw what appeared to be a heavy object bulging in his pocket as he moved. The officers, with over 10 years of foot patrol experience, stepped up and stopped him. They patted him down and found a concealed, unlicensed handgun in the jacket pocket. They then made a full search and found 50 packets of heroin. He filed a motion to suppress the gun and drug evidence. Will the court likely suppress the evidence?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}