{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f5b4cfe-a418-55d7-a3d0-ad7aeb9aa501","task_key":"all--test--8f5b4cfe-a418-55d7-a3d0-ad7aeb9aa501","task_revision_id":"1","upstream_id":"","short_description":"A man and a woman are adjoining landowners in an area of large estates located…","config":"all","split":"test","body":"{\"choices\":\"['The man, because the woman moved onto her property after the stable had been built.', 'The man, because a homeowner is entitled to make reasonable use of his property.', 'The woman, because the noise, coupled with the odors, substantially interfered with the use and enjoyment of her land.', 'The woman, because the man was negligent in permitting the manure to be left unattended.']\",\"question\":\"A man and a woman are adjoining landowners in an area of large estates located in the \\\"chateau\\\" region of southeastern Louisiana. In 2000, the man inherited his estate from his father, whose family had owned the property continuously since 1812. The woman purchased her estate in 2008. The man had a stable, which housed many valuable racing horses, on his property. The stable, built in 2002, stood on a portion of the man's land located only 10 feet from the border of the woman's property. Not infrequently, putrid stenches arose from the man's property, caused by large accumulations of horse manure, which were left unattended. The woman had often complained to the man of the noises and odors emanating from the stable. The woman brought an appropriate action to enjoin the man's use of the stables. Judgment is likely to be for whom?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}