{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7cbe785-0244-5f98-8670-8e3d6a4f7622","task_key":"all--test--a7cbe785-0244-5f98-8670-8e3d6a4f7622","task_revision_id":"1","upstream_id":"","short_description":"A defendant decided to rob a bank. She used an unloaded gun in the robbeiy. The…","config":"all","split":"test","body":"{\"choices\":\"['her gun was unloaded.', \\\"there was not a sufficient connection between the victim's death and the robbery.\\\", \\\"the defendant didn't intend to harm anyone during the robbery.\\\", 'this jurisdiction has adopted the Redline limitation to the felony murder rule.']\",\"question\":\"A defendant decided to rob a bank. She used an unloaded gun in the robbeiy. The defendant approached the bank teller, pointed her unloaded pistol at him and said, \\\"This is a stick-up. . . give me your money and no one will get hurt. \\\"While the teller was handing the money to the defendant, the victim, a bank patron who was in line, saw the defendant pointing the gun at the teller and fainted. He fell backward and cracked his head on the marble floor. This resulted in a fatal head injury. Moments later, the defendant left the bank with the money. Thereafter, the defendant was arrested. If the defendant is prosecuted for felony murder and acquitted, the most likely reason will be because\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}