{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f00fa995-e134-5439-ab19-4d999a1d702c","task_key":"all--test--f00fa995-e134-5439-ab19-4d999a1d702c","task_revision_id":"1","upstream_id":"","short_description":"A defendant is charged with murder and relies upon a claim of self-defense. At…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because the statement is offered to demonstrate its effect on the defendant.', 'admissible, because it qualifies under the state of mind exception to the hearsay rule.', 'inadmissible, because it is hearsay not within any recognized exception.', 'inadmissible, because it is self-serving.']\",\"question\":\"A defendant is charged with murder and relies upon a claim of self-defense. At trial, the defendant attempts to show that prior to the killing, he was told by a drinking buddy that \\\"The victim has killed five men and would as soon kill a man as look at him. \\\"If the statement by the drinking buddy is offered into evidence to prove that the victim, in fact, killed the five men, the statement should be ruled\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}