{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0dbba17d-b41c-5c8d-9df6-6c4944069724","task_key":"all--test--0dbba17d-b41c-5c8d-9df6-6c4944069724","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued a defendant in federal court for assault and battery. At trial,…","config":"all","split":"test","body":"{\"choices\":\"['No, because the witness did not testify at trial.', 'No, unless the court finds, in the interests of justice, that the probative value of the conviction, supported by specific facts and circumstances, substantially outweighs its prejudicial effect.', \\\"Yes, because prior convictions are probative to impeach the witness's character for truthfulness.\\\", 'Yes, because the crime involved an act of dishonesty.']\",\"question\":\"A plaintiff sued a defendant in federal court for assault and battery. At trial, the court has allowed the plaintiff to introduce the deposition testimony of a witness, now deceased, that he was with the plaintiff at the time of the incident. The defendant now seeks to impeach the testimony of the witness with his 13-year-old conviction for burglary (for which he served 18 months in prison) for breaking into the home of a neighbor while she was away and taking some of her valuable jewelry. Should the court allow evidence of the conviction?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}