{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6197219-7cbb-5a01-9243-7fdf5c0d8561","task_key":"all--test--d6197219-7cbb-5a01-9243-7fdf5c0d8561","task_revision_id":"1","upstream_id":"","short_description":"A man entered a half-opened window to a neighbor's trailer and waited in the…","config":"all","split":"test","body":"{\"choices\":\"['No, his belief that she consented in advance was unreasonable and impermissible.', \\\"Yes, rape is a specific intent crime and there's no evidence that he intended a criminal result.\\\", 'Yes, his belief that she consented was reasonable based on prior experience.', 'No, rape of an unconscious woman is a strict liability crime that does not permit a defense.']\",\"question\":\"A man entered a half-opened window to a neighbor's trailer and waited in the dark bedroom inside. After a while, he got in bed and had sexual relations with the woman while she slept. At trial, he claimed that he believed that the woman had consented because he had dated her many times and had consensual sexual relations with her many times in the past. He hadn't seen her in about one week but assumed her consent. He denied specific intent to rape and claimed a desire to surprise her with a \\\"romantic\\\" variation of their usual activities. The woman admitted to prior consensual relations but denied consent on this occasion. Has he stated a viable defense to rape?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}