{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"661b1170-0fb6-5ff7-90ed-96e60a07c3f8","task_key":"all--test--661b1170-0fb6-5ff7-90ed-96e60a07c3f8","task_revision_id":"1","upstream_id":"","short_description":"A man was prosecuted for assault and battery after he admitted striking a victim…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, because it is relevant to show the improbability of the man's having committed an unprovoked attack.\\\", \\\"admissible, because it is relevant to support the man's credibility.\\\", 'inadmissible, because it is merely the opinion of a character witness.', \\\"inadmissible, because it is not directed toward a pertinent trait of the man's character.\\\"]\",\"question\":\"A man was prosecuted for assault and battery after he admitted striking a victim with a pool cue during a barroom argument. The man claimed that he acted in self-defense after he was attacked by the victim, who was drunk and belligerent. As his first defense witness, the man calls his neighbor to testify that the man is a good neighbor. The neighbor's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}