{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"73cc71f3-349d-5a12-84c7-dca5ad7c6145","task_key":"all--test--73cc71f3-349d-5a12-84c7-dca5ad7c6145","task_revision_id":"1","upstream_id":"","short_description":"A defendant was prosecuted for murder. At trial, the prosecutor called a police…","config":"all","split":"test","body":"{\"choices\":\"['plain error, because the trial court should have acted on its own motion to order reference to the lie detector test stricken.', \\\"plain error, because admission of the detective's testimony violated the defendant's privilege against self-incrimination.\\\", \\\"harmless error, because the defendant's trial attorney failed to preserve the argument by timely objection.\\\", \\\"harmless error, because the defendant's statements were admissions.\\\"]\",\"question\":\"A defendant was prosecuted for murder. At trial, the prosecutor called a police detective to testify. The detective, who questioned the defendant at the station house after his arrest, testified that the defendant initially declared his innocence. The detective further testified that when given the opportunity to take a lie detector test, the defendant refused. The defendant's attorney did not object to the detective's testimony. The defendant was convicted and sentenced to prison. The defendant, who is now represented by a new attorney, appeals, claiming that it was error to admit the detective's testimony. The appellate court should find\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}