{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8c67ffe-39b8-52a7-bab5-11463162d4ae","task_key":"all--test--c8c67ffe-39b8-52a7-bab5-11463162d4ae","task_revision_id":"1","upstream_id":"","short_description":"A teenage girl was walking on a street in a retail area. She was carrying a…","config":"all","split":"test","body":"{\"choices\":\"['Yes, there was no reasonable suspicion or probable cause for the officer to stop and search her.', \\\"No, authorities are always allowed to \\\\x80\\\\x98stop and frisk' anyone to see what they are carrying.\\\", 'No, the search was legal because a teenager carrying bags on a downtown street is a suspicious person.', 'Yes, the officer did not get a search warrant even though he had probable cause.']\",\"question\":\"A teenage girl was walking on a street in a retail area. She was carrying a purse and shopping bags. A police officer noticed her and, on a hunch, conducted a random search. He found a variety of jewelry with price tags and no receipts. A jewelry store owner in the vicinity stated that the jewelry was stolen a few minutes earlier, and the girl may have been a customer at that time. She was arrested for theft. The girl's counsel filed a motion to suppress the evidence. Will the judge likely order suppression of the evidence?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}