{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4fcf2463-91e9-5077-856a-efb3c8727925","task_key":"default--test--1251","task_revision_id":"1","upstream_id":"1251","short_description":"A buyer bought a large, nicely kept house near a university campus. She was able…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['\\\"You will be able to recover damages in a nuisance action, because the late-night activities of the fraternity members violate your right to the quiet enjoyment of your property.\\\"', '\\\"You have no nuisance claim, because the fraternity members have the right to use their property as they please.\\\"', '\\\"You will not have a nuisance claim unless you can prove the fraternity\\\\'s activities have caused significant harm to your property.\\\"', '\\\"You have no nuisance claim, as your awareness of the fraternity\\\\'s reputation prior to buying the house implies consent to the noise.\\\"', '\\\"You might have a nuisance claim, but you will need to gather evidence of the disruption caused by the fraternity\\\\'s activities.\\\"', '\\\"You have a strong nuisance claim, as the right to peaceful enjoyment of property overrides any activities of the fraternity.\\\"', '\\\"You might have a nuisance claim, but the fact that you bought the house fully aware of the fraternity\\\\'s habitual late-night activities will count against your claim and could help defeat it.\\\"', '\\\"You cannot make a nuisance claim as the fraternity\\\\'s activities do not exceed the normal bounds of property use.\\\"', '\\\"You have a nuisance claim, as the fraternity\\\\'s activities have disrupted your right to quiet enjoyment of your property, regardless of your prior knowledge.\\\"']\",\"question\":\"A buyer bought a large, nicely kept house near a university campus. She was able to buy the house for a very good price because it was located directly across the street from a fraternity house known for its frequent late-night parties. The buyer knew of the fraternity's reputation before she bought the house. The reputation was well deserved, and the buyer found the noise from the parties extremely unpleasant and disruptive. The buyer has asked an attorney for legal advice regarding a possible nuisance claim against the fraternity. Which of the following responses would best express the applicable law?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}