{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d01eca21-2514-5f2b-ad02-a86a56b449ae","task_key":"all--test--d01eca21-2514-5f2b-ad02-a86a56b449ae","task_revision_id":"1","upstream_id":"","short_description":"A city entered into a small-cost construction contract with a construction…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because it is based on firsthand knowledge.', 'admissible, because it is a report of regularly conducted business activities.', 'inadmissible, because such testimony would be violative of the best evidence rule.', 'inadmissible, because no evidence was introduced to show that the original log records are not available for examination.']\",\"question\":\"A city entered into a small-cost construction contract with a construction company whereby the latter obligated themselves to perform certain minor sewer repair work. Two months after completion of the work, the company sued the city for its failure to pay under the terms of the contract. At trial, the company's construction supervisor who personally oversaw all the repair work was called to testify. He sought to testify about the amount of sewer pipe used, the number of hours required to complete the job, and the number of workers employed by the company during the construction repair work. The defendant's attorney objected on the grounds that the company routinely recorded such factual information in their log books, which were in the company's possession. As a consequence, the supervisor's testimony should be ruled\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}