{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f46f1d5-62fb-53e5-9629-a5a000835c41","task_key":"all--test--9f46f1d5-62fb-53e5-9629-a5a000835c41","task_revision_id":"1","upstream_id":"","short_description":"A defendant is on trial for robbing a particular bank with his best friend. The…","config":"all","split":"test","body":"{\"choices\":\"['No, because inculpatory statements against penal interest do not satisfy the confrontation clause.', \\\"No, because the friend's statement to his mother was not corroborated.\\\", 'Yes, as a statement against penal interest that is not testimonial under the confrontation clause.', \\\"Yes, as a statement of the friend's state of mind that is not testimonial under the confrontation clause.\\\"]\",\"question\":\"A defendant is on trial for robbing a particular bank with his best friend. The friend has left the country and cannot be found. The prosecutor has called the friend's mother to testify to a conversation she had with her son the day before he left the country. She implored him not to go, but he said, \\\"Mom, I have to go. I was involved in a robbery at [the bank], and I don't want them to catch me.\\\" The prosecutor has other evidence indicating that the defendant and the friend were together on the morning of the robbery. Is the statement by the friend to his mother admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}