{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"562a6355-5693-5dfe-b8ec-803e05f5a6db","task_key":"all--test--562a6355-5693-5dfe-b8ec-803e05f5a6db","task_revision_id":"1","upstream_id":"","short_description":"A boyfriend decided to rob a grocery store after he was let go during a labor…","config":"all","split":"test","body":"{\"choices\":\"['1', '2', '3', 'None.']\",\"question\":\"A boyfriend decided to rob a grocery store after he was let go during a labor dispute. The boyfriend asked his girlfriend to drive the getaway car, to which she agreed, on the condition that no loaded weapons were used during the robbery. On the day of the robbery, the boyfriend and his girlfriend drove to the grocery store. Unknown to the girlfriend, the boyfriend entered the store with a loaded gun. The boyfriend approached one of the cashiers and told him to fill a bag with all the money from the register. When the cashier refused, the boyfriend shot and killed him. He then turned to the next cashier and pointed his gun at her. The cashier suffered a heart attack and died. The boyfriend then took the money from the registers himself and exited the store. One of the customers had called the police, and they were waiting outside. A shootout ensued, and the boyfriend was killed by the police. If this jurisdiction follows the agency theory of felony murder, the girlfriend is guilty if how many counts of murder?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}