{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1906e83-0a6d-5ff1-9fe9-a9830e0c95db","task_key":"all--test--a1906e83-0a6d-5ff1-9fe9-a9830e0c95db","task_revision_id":"1","upstream_id":"","short_description":"A man wearing a jacket and jeans walked along a city street at night in a high…","config":"all","split":"test","body":"{\"choices\":\"['No, because the accumulation of factors all added up to reasonable suspicion for a stop and articulable suspicion for a frisk.', 'No, because the police had probable cause to make an arrest when the man began to walk away from them and walk even faster as they followed him.', \\\"Yes, because they didn't know if this was the same man, they observed nothing illegal except jaywalking, and he had a right to avoid them under the circumstances.\\\", 'Yes, because the police lacked reasonable suspicion to stop the man simply because he crossed the street and walked faster away from them.']\",\"question\":\"A man wearing a jacket and jeans walked along a city street at night in a high crime area known for drug trafficking and gun violence. He saw two police officers walking toward him. The officers were checking on a 911 call from a resident who observed a man in jeans and a jacket apparently selling drugs to people on the street. When the police first saw him, he was walking alone, in the area that the woman identified. As they approached, he suddenly crossed the street by jaywalking. They followed, and approaching from behind, asked him to stop. He started walking faster, away from them. The officers saw what appeared to be a heavy object bulging in his pocket as he moved. The officers, with over 10 years of foot patrol experience, stepped up and stopped him. They patted him down and found a concealed, unlicensed handgun in the jacket pocket. They then made a full search and found 50 packets of heroin. He filed a motion to suppress the gun and drug evidence. Will the court likely suppress the evidence?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}