{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30584f13-064b-5a97-bdf1-7f4b2439aa24","task_key":"all--test--30584f13-064b-5a97-bdf1-7f4b2439aa24","task_revision_id":"1","upstream_id":"","short_description":"A shopper suffered injuries to her back and legs when she slipped on a loose…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, because it is relevant to the issue of the owner's knowledge of the unsafe condition.\\\", 'admissible, because it shows that the owner retained ownership and control.', 'inadmissible, because it shows subsequent repair, which is against public policy.', 'inadmissible, because the evidence is irrelevant to the accident.']\",\"question\":\"A shopper suffered injuries to her back and legs when she slipped on a loose piece of carpeting in a hallway connecting a department store and a cafeteria. The shopper brought suit against the owner of the entire building. The owner claimed that since he leased the cafeteria, he no longer retained any control over the connecting hallway. If the shopper offers into evidence the fact that two days after the accident, the owner had the hallway carpet re-laid, the court should rule that the evidence is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}