{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bb00f8da-582d-5c79-bf94-988c665ea570","task_key":"all--test--bb00f8da-582d-5c79-bf94-988c665ea570","task_revision_id":"1","upstream_id":"","short_description":"A woman went to a commercial building for a business meeting. After the meeting,…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the injured person always has the affirmative burden of proving negligence.', 'Yes, because it cannot be presumed that the building owner was negligent.', \\\"No, because the elevator was under the owner's exclusive control and accidents of this nature do not ordinarily occur in the absence of negligence.\\\", 'Yes, because the woman assumed the risk when she decided to use an obviously dangerous instrumentality.']\",\"question\":\"A woman went to a commercial building for a business meeting. After the meeting, she was injured in the building's elevator as it suddenly fell and crashed to the basement from the third floor. She sued the building's owner, but at trial the only evidence she had regarding the issue of negligence was that she was in the elevator when it malfunctioned. Will the trial judge dismiss the case as a matter of law without letting it go to the jury?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}