{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c367540a-4d02-5214-a51e-201f7a620407","task_key":"all--test--c367540a-4d02-5214-a51e-201f7a620407","task_revision_id":"1","upstream_id":"","short_description":"An owner of an apartment building in a rundown section of town knew that one of…","config":"all","split":"test","body":"{\"choices\":\"['recover, because the owner knew that the tenant was dealing drugs from his apartment.', \\\"recover, because the owner was aware that such danger existed because of the tenant's drug-dealing activities.\\\", 'not recover, because the owner is not liable for the criminal acts of third persons.', 'not recover, because the neighbor is outside the zone of foreseeable danger.']\",\"question\":\"An owner of an apartment building in a rundown section of town knew that one of his tenants was dealing drugs from his apartment. He also knew that the tenant's actions had led to some violent confrontations in the past, including one where a gun was discharged in the hallway outside the tenant's apartment. Because the tenant regularly paid his rent on time, the owner didn't take any action to try to evict the tenant. One day, a rival drug dealer drove by the tenant's apartment and fired his gun at the tenant's front window. One of the bullets passed through the tenant's living room and went through the wall into the apartment next door. The bullet struck and injured a neighbor who occupied the adjoining apartment. If the neighbor asserts a claim against the owner to recover damages for his injury, he should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}