{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f77c145-7c38-51b9-8ba6-cc568ea14bbf","task_key":"all--test--5f77c145-7c38-51b9-8ba6-cc568ea14bbf","task_revision_id":"1","upstream_id":"","short_description":"A defendant is on trial for theft of a used car that he took for a test drive…","config":"all","split":"test","body":"{\"choices\":\"['No, because it is a self-serving statement by an accused.', 'No, because it is hearsay not within any exception.', 'Yes, as a prior consistent statement of the defendant.', 'Yes, as a statement by the defendant of his then-existing state of mind.']\",\"question\":\"A defendant is on trial for theft of a used car that he took for a test drive and did not return. He was arrested in the car two days later. In his defense, the defendant has testified that he had no intention of keeping the car but got caught up in marital problems and simply delayed returning it. The defendant calls a witness to testify that the defendant told him, during the two days, \\\"I'm going to return this car as soon as I work things out with my wife.\\\" Is the witness's testimony admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}