{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b4e8213-6d95-5644-913a-ccd3a1b1d9ef","task_key":"all--test--4b4e8213-6d95-5644-913a-ccd3a1b1d9ef","task_revision_id":"1","upstream_id":"","short_description":"During his lunch break one day, an employee took a company vehicle to a…","config":"all","split":"test","body":"{\"choices\":\"['admissible as a declaration against interest.', 'admissible as an admission.', 'inadmissible, because the employee was not unavailable.', \\\"inadmissible, because it violates the employee's Fifth Amendment rights.\\\"]\",\"question\":\"During his lunch break one day, an employee took a company vehicle to a fast-food restaurant and got into an accident on the way back. The driver of the other car sued the company. The company, in its defense, claimed that the employee's use of the vehicle was unauthorized. At trial, the company calls the employee's boss. The boss offers to testify that the employee called him five minutes after the accident and stated, \\\"I hope you won't be sore, because I took the company car without permission. \\\"The court should rule that this evidence is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}