{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e5ee871-02da-53fb-8c6e-cdfdb0dc354f","task_key":"all--test--3e5ee871-02da-53fb-8c6e-cdfdb0dc354f","task_revision_id":"1","upstream_id":"","short_description":"A town had been experiencing a rash of bank robberies. All the witnesses at each…","config":"all","split":"test","body":"{\"choices\":\"['invalidated the lineup.', \\\"violated the defendant's right against self-incrimination.\\\", \\\"tainted the victim's identification of the defendant.\\\", \\\"would not prohibit the defendant's lineup identification from being introduced at trial.\\\"]\",\"question\":\"A town had been experiencing a rash of bank robberies. All the witnesses at each bank had described a man with a gun who said, \\\"Get down on the floor and no one has to die today!\\\" Following a series of leads, the detectives in the case finally made an arrest. At the police station, the suspect was placed in a lineup. One of the victims observed the lineup, and after each of the six participants stated, \\\"Get down on the floor and no one has to die today,\\\" the victim identified the suspect as the perpetrator. The defendant's attorney subsequently moved to suppress the out-of-court identification, arguing that because the defendant was forced to speak, this violated his Fifth Amendment right to remain silent, which had been invoked previously by the defendant. The police requirement that the defendant state at the lineup, \\\"Get down on the floor and no one has to die today\\\"\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}