{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89a315fa-cb7e-5428-97b5-278175f88f40","task_key":"all--test--89a315fa-cb7e-5428-97b5-278175f88f40","task_revision_id":"1","upstream_id":"","short_description":"A man and a woman agreed that the woman would rob a bank and that the man would…","config":"all","split":"test","body":"{\"choices\":\"['No, because the man received no benefit from the robbery.', 'No, because the man withdrew from the conspiracy.', \\\"Yes, because the robbery was successful due in part to the man's actions.\\\", 'Yes, because there was an agreement to rob the bank and an overt act in furtherance of the agreement.']\",\"question\":\"A man and a woman agreed that the woman would rob a bank and that the man would steal a car beforehand for the woman to use as a getaway vehicle. The man stole a car and parked it two blocks from the bank. He left the car key under the floor mat with a note saying that he wanted nothing more to do with the scheme. The next day, the woman robbed the bank and ran to the spot where the man had said he would leave the stolen car. She then escaped in the car and disappeared. She never shared any of the money with the man. In a jurisdiction that has adopted the bilateral requirement for conspiracy, can the man properly be convicted of conspiring with the woman to rob the bank?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}