{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2a7cebcd-5e48-5167-af84-cbdc6f486b9c","task_key":"all--test--2a7cebcd-5e48-5167-af84-cbdc6f486b9c","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued a defendant over title to land on a riverbank. Changes in the…","config":"all","split":"test","body":"{\"choices\":\"['No, because they are hearsay not within any exception.', 'No, because they violate the \\\"best evidence\\\" rule.', 'Yes, as summaries of voluminous business records.', \\\"Yes, as the draftsman's expert opinion of the water levels.\\\"]\",\"question\":\"A plaintiff sued a defendant over title to land on a riverbank. Changes in the water level over time were important to the plaintiff's case. For 15 years, a commercial fisherman had kept a daily log of the water level at his dock on the riverbank opposite the land in order to forecast fishing conditions. The plaintiff hired a draftsman to graph the data from the fisherman's logs for use as a trial exhibit. At trial, the fisherman testified to the care with which he had made the measurements and recorded them in his logs, which had been made available for the defendant's inspection. The draftsman then testified to the manner in which he had prepared the graphs. With this foundation, are the graphs admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}