{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9dd8aa6d-a4d1-5fd5-91bb-036c930939f1","task_key":"all--test--9dd8aa6d-a4d1-5fd5-91bb-036c930939f1","task_revision_id":"1","upstream_id":"","short_description":"A driver has been charged with leaving the scene of an accident. The driver…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because it is the statement of the witness herself, who is subject to cross-examination.', 'admissible, because it is a statement based on firsthand knowledge.', 'inadmissible, because the bystander is unavailable and, therefore, cannot contradict the driver.', 'inadmissible, because it is hearsay not within any recognized exception.']\",\"question\":\"A driver has been charged with leaving the scene of an accident. The driver allegedly hit another car in a parking lot and drove away without leaving proper insurance and contact information. A bystander wrote down her license plate number and called the police. At trial, the driver proposes to testify that the day after the accident, she met with the police officer investigating the accident, and told him that she did not leave until after she had inspected both cars and determined there was no damage. The bystander is not available to testify at trial. The driver's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}