{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4508f15-f466-5560-91bb-c03e849f1deb","task_key":"all--test--b4508f15-f466-5560-91bb-c03e849f1deb","task_revision_id":"1","upstream_id":"","short_description":"A campaign worker for a City Council candidate distributed a flier saying that…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because legal impossibility is a defense to the crime of conspiracy.', 'Yes, because he could not be guilty of conspiring with two law enforcement agents who never intended to commit the crime.', 'No, because the basis of the conspiracy charge is the agreement to commit the act, along with an affirmative step in that direction.', 'No, because when the worker drove the agents to the voting place, he committed the complete crime of bribery and the conspiracy was an element contained within it.']\",\"question\":\"A campaign worker for a City Council candidate distributed a flier saying that the candidate was offering transportation to the polls on election day. The flier stated that recipients should come to a nearby storefront on election day, where there would be free food and drink, and other \\\"prizes\\\" for those who voted. Two police undercover agents went to the party, where they agreed to be taken to vote in return for $50 each. The campaign worker drove the men to their alleged poll location, where the agents arrested him on charges of conspiracy to violate the election bribery laws. The defense filed a motion to dismiss on the basis that it was legally impossible to commit the crime because the agents were not registered to vote in that district and they never intended to allow a crime to be permitted. Is the trial court likely to grant the motion to dismiss?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}