{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40ce2336-109b-59f3-8e79-2ec2f805242c","task_key":"all--test--40ce2336-109b-59f3-8e79-2ec2f805242c","task_revision_id":"1","upstream_id":"","short_description":"A restaurant building was expanded into a vacant lot north of its original…","config":"all","split":"test","body":"{\"choices\":\"['The neighboring property owner, because he should have been given notice of the intrusion before it occurred.', 'The neighboring property owner, because the restaurant intruded upon his property the restaurant intruded upon his property without permission.', 'The restaurant owner, because the decision to provide the additional support was reasonable.', 'The restaurant owner, because there was no disturbance of peaceful enjoyment of the neighboring property. \\\\x80\\\\x83']\",\"question\":\"A restaurant building was expanded into a vacant lot north of its original location. The contractor who was working on the restaurant expansion found that the north wall of the new structure needed extensive support, so anchor rods and concrete were added. The supporting anchor rods and concrete extended into a neighboring property farther to the north at a point 20 feet below the surface. Although there was no impact on the surface of his land or on existing uses, the owner of the neighboring property has sued the restaurant owner for trespass. Which party is likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}