{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e022362-691c-5b3a-964e-54fee3eef0fd","task_key":"all--test--6e022362-691c-5b3a-964e-54fee3eef0fd","task_revision_id":"1","upstream_id":"","short_description":"A gardener sued a homeowner for breach of contract. The gardener claims that the…","config":"all","split":"test","body":"{\"choices\":\"['admissible as an admission by a party-opponent.', 'admissible as non-hearsay to corroborate evidence of a verbal act.', \\\"inadmissible, because of the Dead Man's Statute.\\\", \\\"inadmissible, unless it is established that the gardener accepted the friend as the homeowner's agent in communicating the terms of the contract.\\\"]\",\"question\":\"A gardener sued a homeowner for breach of contract. The gardener claims that the homeowner hired him to perform landscape work at his home. After performing the gardening services, the homeowner paid him $100. The gardener, who speaks only English, contends that the homeowner, who speaks only Japanese, originally agreed to pay him $200 for the work. When the parties entered into their agreement, the gardener was accompanied by his friend, who is now deceased, who spoke both English and Japanese. This jurisdiction has a typical Dead Man's Statute in effect. At trial, the gardener now seeks to testify to the terms of his agreement with the homeowner as communicated to him by his friend. Upon objection by the homeowner's attorney, the gardener's proposed testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}