{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"522e0a35-6bda-5839-ae35-3ec198b9d064","task_key":"all--test--522e0a35-6bda-5839-ae35-3ec198b9d064","task_revision_id":"1","upstream_id":"","short_description":"A college student owns an expensive sports car. His friend called him up one…","config":"all","split":"test","body":"{\"choices\":\"['admissible as evidence of habit.', 'admissible as a present sense impression.', \\\"inadmissible, because it goes to the college student's character.\\\", 'inadmissible, because she is biased.']\",\"question\":\"A college student owns an expensive sports car. His friend called him up one afternoon and asked to borrow his car for a date he had that night. The college student generously lent the car to his friend. On his way home from the date, basking in the memories of a beautiful evening, the friend lost control of the car and hit a pedestrian. As a result of the accident, and after learning that the friend had no money and no insurance, the pedestrian brought suit against the college student to recover for his injuries. He asserted a claim against the college student for negligent entrustment of a vehicle. During his case, the pedestrian introduced evidence that the friend had been in several previous accidents. In his defense, the college student offered the testimony of his girlfriend that the college student frequently lent his car to his friend. The girlfriend further testified that the college student gave his friend permission to use his car only after determining that the friend was a careful and attentive driver. The girlfriend's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}