{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51167b26-51af-52f1-b6d3-e671bc496d77","task_key":"all--test--51167b26-51af-52f1-b6d3-e671bc496d77","task_revision_id":"1","upstream_id":"","short_description":"A man took money without permission from his employer to compensate himself for…","config":"all","split":"test","body":"{\"choices\":\"['The charges will not be dismissed because his intent becomes irrelevant when he uses force to obtain possession of property.', 'The charges will be dismissed because the intent to steal is a required element of both theft and larceny and this man did not intend to steal because he thought that he had a right to the money.', 'The charges will be dismissed because he can raise the defense of necessity and it is always lawful to take what a person needs to survive.', \\\"The charges will not be dismissed because the man's right to the money was disputed and therefore his intent is irrelevant as compared to the employer's intent and belief.\\\"]\",\"question\":\"A man took money without permission from his employer to compensate himself for wages he honestly believed were owed to him. The authorities arrested the man for felonious larceny, a crime that is also called theft in some states. The man defended on the basis that he had no intent to steal and only wanted his rightful compensation. He filed a motion to dismiss, relying essentially on proof that he had no intent to steal because he believed in good faith that it was his own property. What will be the most likely decision of the court?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}