{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22fc6dff-8487-5ab1-8a10-0ef896c819a7","task_key":"all--test--22fc6dff-8487-5ab1-8a10-0ef896c819a7","task_revision_id":"1","upstream_id":"","short_description":"In 1996, an investor purchased a 100-acre tract located in a northern county of…","config":"all","split":"test","body":"{\"choices\":\"['the residency restriction is no longer enforceable because of the change in the character of the neighborhood surrounding the development.', 'the residency restriction is no longer enforceable because the area surrounding the development was rezoned for commercial use.', 'the residency restriction is enforceable, thus preventing any commercial use.', 'the operation of a beauty parlor would not constitute a violation of the residency restriction.']\",\"question\":\"In 1996, an investor purchased a 100-acre tract located in a northern county of a state. Shortly thereafter, the investor prepared a subdivision plan that created 100 one-acre residential building lots on this tract. In 1997, the subdivision plan was recorded with the county recorder's office. During the next few years, the investor sold all 100 residential lots to individual purchasers. Each deed specified that every lot designated on the subdivision plan was to be recorded in the county recorder's office. Each deed also provided the following: \\\"No lot shall be used except for residential purposes. \\\" By 2009, the area surrounding the subdivision was rezoned for commercial and business uses. One of the lot owners now decides to operate a small beauty parlor in the basement of her home. In an action by the homeowners in the subdivision to prevent such commercial use, the court will most likely hold that\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}