{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6428549-aa65-5cb5-b694-e8d4bdd83a74","task_key":"all--test--a6428549-aa65-5cb5-b694-e8d4bdd83a74","task_revision_id":"1","upstream_id":"","short_description":"While shopping at a mall, a woman suffered injuries when an escalator…","config":"all","split":"test","body":"{\"choices\":\"['inadmissible, because it is irrelevant.', 'inadmissible, because it is not known whether the other shopper the woman was exercising reasonable care in riding the escalator.', \\\"admissible, because it tends to prove that a dangerous condition was present at the time the woman's foot was crushed in the escalator.\\\", \\\"admissible, because of its probative value in establishing that the woman's injury was caused by the malfunctioning escalator.\\\"]\",\"question\":\"While shopping at a mall, a woman suffered injuries when an escalator malfunctioned, crushing her foot. The woman brings suit against the mall. At trial, she calls another shopper to testif' that he rode the same escalator just moments before the woman's accident and had to jump in the air at the last minute to avoid getting his foot crushed in the escalator. This testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}