{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fae2d979-a8b0-5090-9b35-cf32219b164e","task_key":"all--test--fae2d979-a8b0-5090-9b35-cf32219b164e","task_revision_id":"1","upstream_id":"","short_description":"One day, while riding his motorcycle through a residential area, a man hit and…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, on the issue of the defendant's negligence.\\\", 'admissible, to show that the defendant is able to pay for the medical expenses.', 'inadmissible, because the testimony was highly prejudicial.', 'admissible, only to prove ownership or control of the motorcycle.']\",\"question\":\"One day, while riding his motorcycle through a residential area, a man hit and injured a child who darted into his path. The child's mother files suit against the man. At trial, the mother is then called to testify as to her conversation with the man at the hospital where her son was rushed immediately following the accident. According to the mother, the man told her, \\\"Don't worry, my insurance company will pay for all of your son's medical expenses. \\\"If offered into evidence, the court will most likely rule that the mother's statement is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}