{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5fdaf752-1332-5b01-9b50-8c41a0e14951","task_key":"all--test--5fdaf752-1332-5b01-9b50-8c41a0e14951","task_revision_id":"1","upstream_id":"","short_description":"Having just stolen a car, a defendant was operating the car within the posted…","config":"all","split":"test","body":"{\"choices\":\"['correct, because malice can be inferred from the intent to steal.', 'correct, because malicious destruction of property is a general intent crime.', 'incorrect, because larceny is not a lesser included offense of malicious destruction of property.', 'incorrect, because malice requires a showing of recklessness.']\",\"question\":\"Having just stolen a car, a defendant was operating the car within the posted speed limit when the steering suddenly malfunctioned. The car swerved out of control, jumped the curb, and crashed into a home. The defendant was arrested and charged with the separate crimes of larceny and malicious destruction of property. During the trial the judge instructed the jury that the theft of the car was sufficient evidence to satisfS' the malice requirement for the malicious destruction of property charge. The defendant's attorney has filed a motion objecting to the judge's jury instruction. The judge's instruction was\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}