{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0765e2ad-b84d-588a-9322-193a89f7459a","task_key":"all--test--0765e2ad-b84d-588a-9322-193a89f7459a","task_revision_id":"1","upstream_id":"","short_description":"A customer is suing a car dealer for selling him a salvaged vehicle that the car…","config":"all","split":"test","body":"{\"choices\":\"['admissible as a statement of then-existing mental or emotional condition.', 'admissible as a declaration against interest.', 'admissible as a statement of present sense impression.', 'inadmissible as hearsay not within any recognized exception.']\",\"question\":\"A customer is suing a car dealer for selling him a salvaged vehicle that the car dealer had represented as being brand new. A few weeks before trial, the car dealer approached his sister and said, \\\"Sis, I need some sympathy. I sold a salvaged vehicle to a customer, and now he's suing me. I didn't mean any harm by it. I inspected the vehicle and everything. \\\" Unknown to either the car dealer or his sister, the sister's boyfriend was at the front door and overheard this conversation. When the time for trial came around, the car dealer left the country and refused to attend the trial, telling his attorney to handle it. The customer's attorney attempted several times to secure the car dealer's attendance at trial, but was unsuccessful. At trial, the sister's boyfriend is called to testify about the conversation he overheard. On objection by the car dealer's attorney, the court will most likely rule the testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}