{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af09de5a-5011-566f-a089-3d806cb2f442","task_key":"all--test--af09de5a-5011-566f-a089-3d806cb2f442","task_revision_id":"1","upstream_id":"","short_description":"A defendant worked as a short-order cook at a restaurant. After work, the…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the cook and the defendant did enter into an agreement to commit the bank robbery.', 'Yes, because the purchase of the ski masks was a sufficient overt act in the furtherance of the crime.', 'Yes, because the defendant did not thwart the cook from committing the robbery.', \\\"No, because the defendant's withdrawal was effective.\\\"]\",\"question\":\"A defendant worked as a short-order cook at a restaurant. After work, the defendant went out drinking with one of the other cooks, and they were commiserating about their lack of money. They decided that the only way out of their rut was to rob a local bank. They then agreed to carry out a bank heist on the following Friday. In accordance with their plan, the cook on Monday purchased two ski masks to be used in the robbery. On Wednesday, after learning that the cook had bought the masks, the defendant got cold feet and told the cook he was renouncing his involvement in the robbery scheme. On Friday, the cook carried out the robbery without the defendant's participation. This jurisdiction requires an overt act for the crime of conspiracy. Is the defendant guilty of conspiracy?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}