{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"124eedf4-4534-5a2f-8f29-577ad59a7160","task_key":"all--test--124eedf4-4534-5a2f-8f29-577ad59a7160","task_revision_id":"1","upstream_id":"","short_description":"A defendant was charged with attempted rape of a victim. The crime allegedly…","config":"all","split":"test","body":"{\"choices\":\"['guilty, because consent is determined by the objective manifestations of the victim and not the subjective beliefs of the defendant.', 'guilty, because voluntary intoxication is no defense.', 'not guilty, because he honestly believed that she was consenting.', 'not guilty, because his belief that she was consenting was reasonable.']\",\"question\":\"A defendant was charged with attempted rape of a victim. The crime allegedly occurred at a party at the defendant's home. During the party, the defendant invited the victim into his bedroom to show her his tattoos. When she entered his bedroom, the defendant ripped off her blouse and threw her onto his bed. He then jumped on the victim and tried to pull off her skirt. When the victim began to scream, some of the guests rushed into the bedroom and pulled the defendant off the victim. At trial, the defendant testified that he wanted to have sexual intercourse with the victim but he believed that she was consenting. The defendant further testified that he had consumed a pint of whiskey earlier in the evening and was intoxicated at the time the incident occurred. If the jury believes that the victim did not consent but also believes that the defendant, in his intoxicated state, honestly believed that she was consenting, the defendant should be found\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}