{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"37ac934f-bd24-5854-aa28-6854bb7a7bf6","task_key":"all--test--37ac934f-bd24-5854-aa28-6854bb7a7bf6","task_revision_id":"1","upstream_id":"","short_description":"A woman was in a public restroom at a mall. While the woman was at the sink, a…","config":"all","split":"test","body":"{\"choices\":\"['Assault.', 'Battery.', 'Assault with intent to commit rape.', 'No crime.']\",\"question\":\"A woman was in a public restroom at a mall. While the woman was at the sink, a man entered the restroom and told her not to make a sound. Frightened, the woman ran to an open window in the bathroom and jumped onto the fire escape. As she exited down the fire escape, the woman accidentally fell, bruising her legs. Which of the following crimes, if any, would the man most likely be convicted of?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}