{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"675ff0ca-986b-50bc-b7f6-113e728e8356","task_key":"all--test--675ff0ca-986b-50bc-b7f6-113e728e8356","task_revision_id":"1","upstream_id":"","short_description":"A young man suggested to his friend that they steal a large-screen TV from a…","config":"all","split":"test","body":"{\"choices\":\"['No crime.', 'Conspiracy.', 'Burglary.', 'Conspiracy and larceny.']\",\"question\":\"A young man suggested to his friend that they steal a large-screen TV from a neighbor's house. The friend wasangry with the young man and decided to use the opportunity to get even with him by having him arrested. Thefriend said he would help, and that night, he drove the young man to the neighbor's house. The young manbroke in while the friend remained outside. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The friend is guilty of what offense in a common law jurisdiction?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}