{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bf16cd0a-8037-5164-9158-be5c4582f9b9","task_key":"all--test--bf16cd0a-8037-5164-9158-be5c4582f9b9","task_revision_id":"1","upstream_id":"","short_description":"A defendant and his friend were walking down the street when they ran into a…","config":"all","split":"test","body":"{\"choices\":\"['Attempted murder.', 'Murder.', 'Assault with a deadly weapon.', 'Concealment of a deadly weapon.']\",\"question\":\"A defendant and his friend were walking down the street when they ran into a victim. The friend turned to the defendant and said that the victim owed him some money. The friend then stopped the victim and demanded the money. The victim refused to pay the money. The friend suddenly said to the defendant, \\\"Give me your gun. I'm going to blow this lowlife away. \\\" The defendant gave his friend his gun, and the friend shot the victim to death. As the friend and the defendant were about to leave, the defendant turned to his friend and said, \\\"Let me have my gun back. I think the creep's still alive. \\\" The defendant then fired two more shots into the victim's body. Unknown to the defendant, the victim was already dead. Which of the following is the most serious crime that the defendant can be convicted of?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}