{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dff3e3c8-d830-58ec-bf55-ec513f973cc9","task_key":"all--test--dff3e3c8-d830-58ec-bf55-ec513f973cc9","task_revision_id":"1","upstream_id":"","short_description":"A young man suggested to his friend that they steal a large-screen TV from a…","config":"all","split":"test","body":"{\"choices\":\"['No crime.', 'Conspiracy.', 'Burglary.', 'Conspiracy and larceny.']\",\"question\":\"A young man suggested to his friend that they steal a large-screen TV from a neighborÕs house. The friend was angry with the young man and decided to use the opportunity to get even with him by having him arrested. The friend said he would help, and that night, he drove the young man to the neighborÕs house. The young man broke in while the friend remained out-side. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The jurisdiction defines crimes as at common law. Of what crime, if any, can the friend properly be convicted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}