{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c3b3dd2-9b46-5977-b145-dddcbed824de","task_key":"all--test--2c3b3dd2-9b46-5977-b145-dddcbed824de","task_revision_id":"1","upstream_id":"","short_description":"A baseball card dealer agreed to sell an extremely rare card to a collector. The…","config":"all","split":"test","body":"{\"choices\":\"['be to his advantage to claim that the courier was his accomplice and was participating in the scheme to receive a share of the proceeds.', 'be to his advantage to claim that the courier knew nothing of the scheme but was simply hired by the dealer to make the delivery.', 'be to his advantage to claim that the courier knew nothing of the scheme but was simply hired by the collector to make the delivery.', \\\"make no difference whether the courier was the dealer's accomplice or was an innocent agent.\\\"]\",\"question\":\"A baseball card dealer agreed to sell an extremely rare card to a collector. The collector lived in another state. As part of their agreement, a courier was hired to pick up the card from the dealer and transport it to the collector. The courier was then to get the purchase money from the collector and bring it back to the dealer. But rather than entrusting the genuine card to the courier, the dealer gave him a counterfeit replica that was valueless. After picking up the envelope containing the replica, the courier left the dealer's home and embarked on his trip. The police, however, received a tip concerning the scheme and intercepted the courier's car before he left the city limits. Thereafter, the dealer was arrested and charged with attempting to obtain property by false pretenses. If the dealer contends that the scheme had not progressed far enough to constitute an attempt, it will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}