{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d0813cc-f55a-56e4-83ea-17a656da570c","task_key":"all--test--1d0813cc-f55a-56e4-83ea-17a656da570c","task_revision_id":"1","upstream_id":"","short_description":"The police arrested a man for the murder of his live-in fiancée after the…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Yes, because the sister does not have sufficient experience and knowledge to be able to identify the man's voice and to say that the voice and the defendant were the same person.\\\", 'Yes, because the conversation is hearsay and there are no exceptions that would allow it into evidence.', \\\"No, because there was sufficient circumstantial evidence to make a reasonable conclusion that the man she talked to was her sister's fiancé.\\\", 'No, because telephone conversations do not need to be authenticated, because there is a presumption that the person talking is the same person who owns the phone number.']\",\"question\":\"The police arrested a man for the murder of his live-in fiancée after the woman's dead body was found in the basement of their apartment building. At trial, the prosecution attempted to present the testimony of the victim's out-of-state sister regarding a telephone conversation she had with the defendant. She had called their landline number to talk with her sister on the same day when he had first reported to the police that his fiancé had been missing for five days. The sister had not met the man but had spoken with him briefly a few other times. He told her that her sister \\\"went out all night last night and did not return yet.\\\" The prosecution attempted at trial to have the sister testify to the conversation to show the significant discrepancy between his stories to the police and to her. The defense objected, arguing that she couldn't authenticate the conversation because she couldn't identify the speaker as the defendant nor had she ever met him. The prosecution told the court that she felt it was \\\"probably\\\" his voice, which she recollected from the prior talks at the same number. Will the court likely grant the defendant's objection to the testimony?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}